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Preface 


The  U.S.  Air  Force’s  remotely  piloted  aircraft  (RPAs)  have  played  a  significant  role  in  current 
operations  in  Southwest  Asia.  As  the  inventory  of  RPAs  increases  and  new  sensor  technologies 
come  online  in  the  coming  years,  the  Air  Force  has  an  opportunity  to  consider  additional  roles 
for  these  aircraft.  Thoughtful  study  into  these  possibilities  will  ensure  that,  when  the  Air  Force 
employs  RPAs,  they  will  help  fill  capability  gaps  or  augment  existing  capabilities  in  more- 
efficient  or  more-effective  ways. 

The  purpose  of  this  documented  briefing  is  to  describe  a  suite  of  tools  developed  by 
RAND  Project  AIR  FORCE  (PAF)  to  help  the  Air  Force  think  through  future  roles  for  RPAs. 
It  describes  tools  to  evaluate  platform  selection  and  concept  of  operations  (CONOPS)  develop¬ 
ment,  sensor  performance  against  various  targets,  weapon  effects,  environmental  factors,  plat¬ 
form  survivability,  computational  processing  of  data,  and  exploitation  of  sensor  products.  This 
document  also  explains  how  the  separate  analysis  in  each  of  these  areas  feeds  into  a  mission- 
level  analysis,  performed  with  PAF’s  Systems  and  CONOPS  Operational  Effectiveness  Model 
(SCOPEM),  and  a  campaign-level  analysis  using  PAF’s  Force  Structure  Effectiveness  (FSE) 
model.  Use  of  these  tools  and  models  will  help  clarify  how  future  RPAs  can  contribute  to 
U.S.  warfighting  in  cost-effective  ways.  The  tools  presented  here  are  also  useful  for  examining 
the  effectiveness  of  new  capabilities  more  broadly  (e.g.,  directed  energy  weapons  or  electronic 
warfare  capabilities);  examining  the  effectiveness  of  new  platforms  in  the  context  of  the  entire 
intelligence,  surveillance,  and  reconnaissance  (ISR)  force  posture;  and  evaluating  the  most 
cost-effective  ISR  force  structure  to  meet  future  operational  needs. 

The  model  development  and  analytic  approaches  documented  here  should  be  of  interest 
to  Air  Force  personnel  involved  in  future  system  acquisition,  force  structure  planning,  and 
studies  and  analyses.  They  should  also  be  of  interest  to  the  growing  RPA  community  within 
the  Air  Force  and  the  intelligence  officers  who  grapple  with  the  increasing  demands  for  pro¬ 
cessing,  exploitation,  and  dissemination  (PED). 

The  research  reported  here  was  sponsored  by  Randall  Walden,  Director  for  Information 
Dominance  Programs,  Office  of  the  Assistant  Secretary  of  the  Air  Force  for  Acquisition;  Lt  Gen 
David  Deptula,  then-Deputy  Chief  of  Staff  for  Intelligence,  Surveillance  and  Reconnaissance, 
Headquarters  U.S.  Air  Force;  and  Maj  Gen  Thomas  Andersen,  then  Director  of  Requirements, 
Headquarters  Air  Combat  Command,  and  conducted  within  the  Force  Modernization  and 
Employment  Program  of  RAND  Project  AIR  FORCE  for  a  fiscal  year  2010  study,  “Develop¬ 
ing  an  Analytically  Based  Vision  for  Air  Force  Capabilities  Delivered  by  Remotely  Piloted  Air¬ 
craft.”  A  companion  report  explores  the  suitability  of  RPAs  to  help  meet  capability  gaps  in  the 
2009  Air  Force  Capabilities  Review  and  Risk  Assessment  (Lingel  et  ah,  2011). 
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Summary 


The  U.S.  Air  Force’s  remotely  piloted  aircraft  (RPAs),  such  as  RQ-4  Global  Hawk,  MQ-1  Pred¬ 
ator,  and  MQ-9  Reaper,  have  made  significant  contributions  to  current  operations  in  South¬ 
west  Asia.  Although  these  aircraft  primarily  provide  intelligence,  surveillance,  and  reconnais¬ 
sance  (ISR)  to  joint  warfighters,  armed  variants  are  also  able  to  provide  rapid  precision  strike 
against  time-sensitive  targets.  The  planned  increase  in  RPA  inventories  over  the  next  several 
years  reflects  a  growing  awareness  of  the  value  of  these  aircraft.  In  addition,  new  sensor  tech¬ 
nologies,  such  as  wide-area  airborne  surveillance  (WAAS),  will  further  augment  RPAs’  poten¬ 
tial  contribution  to  future  warfighting. 

Given  these  developments,  the  time  is  ripe  for  the  Air  Force  to  consider  additional  roles 
for  future  RPAs,  whether  to  help  address  capability  gaps  that  are  currently  unfulfilled  or  to 
replace  or  complement  manned  systems  in  current  missions.  Thoughtful  study  is  needed  to 
identify  promising  mission  areas,  to  consider  potential  platform  alternatives,  and  to  analyze 
how  different  options  could  contribute  to  specific  missions  and  to  overall  campaigns  in  cost- 
effective  ways. 

This  documented  briefing  discusses  a  suite  of  tools  and  models  developed  by  RAND  Proj¬ 
ect  AIR  FORCE  (PAF)  researchers  to  help  the  Air  Force  think  through  these  issues.  Figure  S.l 
depicts  the  overall  methodology  for  evaluating  operational  effectiveness.  When  analyzing 
alternative  force  structures,  the  first  step  is  to  identify  capability  gaps.  Sources  may  include  the 
Air  Force  Capabilities  Review  and  Risk  Assessment  and  the  Multi-Service  Force  Deployment 
scenarios.  The  next  step  is  to  develop  appropriate  mission  vignettes  that  represent  a  range  of 
ways  in  which  platforms  could  help  fill  capability  gaps.  Next,  the  trade-offs  between  differ¬ 
ent  candidate  RPAs  and  other  systems  (such  as  manned  platforms,  satellites,  or  ground-based 
systems)  should  be  explored  within  a  series  of  mission  vignettes.  The  vignettes  are  selected  to 
represent  a  range  of  potential  future  RPA  roles  under  a  range  of  conditions.  Finally,  the  analysis 
results  in  an  effectiveness  matrix,  which  describes  the  conditions  under  which  different  plat¬ 
forms  and  configurations  are  effective.  The  operational  effectiveness  of  a  system  can  be  defined 
as  the  degree  to  which  it  improves  the  warfighter’s  level  of  success  in  a  given  set  of  missions  or 
enlarges  the  range  of  conditions  under  which  the  warfighter  is  likely  to  be  successful  in  those 
missions.  Operational  effectiveness  cannot  be  computed  from  the  technical  specifications  of 
systems  alone  but  can  be  observed  only  in  terms  of  outcomes  in  an  operational  context  that 
includes  all  the  other  capabilities  in  the  theater,  including  space  and  threat  systems.  Because 
many  or  all  of  the  systems  being  compared  do  not  yet  exist,  placing  the  system  in  an  opera¬ 
tional  context  requires  constructive  simulation.  Simulation  further  allows  examination  of  a 
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Figure  S.1 

Remotely  Piloted  Aircraft  Evaluation  Methodology 
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breadth  of  situations  within  a  relatively  short  time  frame.1  The  effectiveness  matrix  provides 
insights  into  the  operational  suitability  of  a  given  system  at  the  mission  level.  In  order  to  under¬ 
stand  the  overall  force  structure  needed,  analysis  at  the  campaign  level  must  occur.  The  result 
provides  understanding  of  a  candidate  system’s  performance  based  on  campaign-level  mea¬ 
sures  of  effectiveness  (MOEs). 

Additional  entry  points  exist  for  leveraging  PAF’s  analytic  methodology.  For  example, 
alternative  candidate  RPAs  or  advanced  technologies  may  be  proposed  for  evaluation.  In  this 
instance,  the  evaluation  would  flow  from  the  “Define  candidate  RPA  systems”  step  to  the 
capability-gap  step  in  order  to  identify  which  gaps  the  system  or  technology  might  address. 
Another  possibility  would  be  evaluating  force  structure  alternatives  (based  on  budget  con¬ 
straints)  in  which  the  study  objective  is  to  assess  proposed  force  structures  and  the  analysis 
would  flow  to  identifying  the  capability  gaps  that  could  arise  for  each.  Fast,  analysis  could  start 
with  the  mission  vignette  step,  in  which  the  Air  Force  wants  an  assessment  of  the  impact  of  a 
new  threat  capability,  for  example. 

The  models  and  tools  used  in  this  methodology  are  depicted  in  Figure  S.2.  Their  pur¬ 
pose  is  to  analyze  the  operational  effectiveness  and  cost-effectiveness  of  RPAs  and  other  can¬ 
didate  systems  (including  space  systems  and  ground-based  assets).  Systems  must  be  evaluated 
at  the  mission  level  first  for  operational  effectiveness  and  then  at  the  campaign  level  to  deter¬ 
mine  overall  Force  Structure  Effectiveness  (FSE).  The  suite  includes  tools  to  analyze  specific 
aspects  of  platforms,  sensor  performance  against  various  targets,  weapon  effects,  environmen¬ 
tal  factors,  platform  survivability,  weapon  employment,  computational  processing  of  data,  and 
exploitation  of  sensor  products.2  These  individual  tools  contribute  to  the  mission-level  analysis 


1  Constructive  simulation  is  typically  a  time-  or  event-stepped  abstraction  of  force-on-force  operations,  often  employing 
digital  terrain  for  analysis. 

2  The  reader  is  referred  to  textbooks  for  sensor  details,  such  as  Introduction  to  Radar  Systems  by  Merrill  Skolnik  and  Intro¬ 
duction  to  Sensor  Systems  by  Shahen  Hovanessian. 
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Figure  S.2 

RAND's  Systems  and  CONOPS  Operational  Effectiveness  Model 


SCOPEM 

Platform 

•  Orbit,  flight  path 

Environment 

•  Speed,  altitude 

•  Terrain  (LOS) 

•  CONOPS 

•  Sea  state,  wind 

•  Atmosphere 

Sensors 

•  Space  weather 

•  NIIRS  capabilities  (EO,  IR,  SAR,  FMV) 

•  RCS  resolution  (ISAR,  MTI) 

Survivability 

•  Sensitivity,  scan  cycle  (SIGINT) 

•  Air-to-air  threats 

Weapons 

•  Integrated  air  defenses 

•  Range  and  speed 

•  Accuracy  (CEP) 

Processing 

•  Onboard  processing  (SWaP) 

Targets 

•  Communications,  PED  delays 

•  NIIRS  characteristics  (EO,  IR,  SAR) 

•  RCS  (ISAR,  MTI) 

•  ERP,  shape,  frequency  (SIGINT) 
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performed  with  PAF’s  Systems  and  CONOPS  [concept  of  operations]  Operational  Effective¬ 
ness  Model  (SCOPEM).  SCOPEM  is  structured  to  build  a  rich  vignette,  including  terrain 
effects,  multiple  assets  operating  together,  varied  behaviors  for  platforms,  and  other  features 
that  simulate  complex  operational  environments.  The  modeling  occurs  in  simple  modules  of 
code,  which  provide  insight  into  the  factors  that  drive  mission-level  outcomes.  MOE  examples 
derived  from  SCOPEM  modules  include  detection  of  a  target  for  a  sensor,  line-of-sight  (LOS) 
obscuration  from  terrain,  and  probability  of  kill  from  weapon  employment.  This  level  of  detail 
is  essential  to  building  an  effectiveness  matrix,  which  not  only  identifies  effective  platforms 
and  CONOPS  but  also  defines  the  range  of  conditions  under  which  platforms  either  succeed 
or  fail  at  a  given  mission. 

It  is  not  enough  to  evaluate  a  platform’s  mission  effectiveness  to  pursue  a  given  candidate 
platform;  one  must  also  understand  force  structure  implications  of  employing  a  particular 
RPA.  To  do  so,  we  explore  RPA  employment  at  the  campaign  level.  For  example,  we  would  like 
to  know,  for  a  given  distribution  of  targets  over  an  area  of  responsibility  (AOR),  what  the  effec¬ 
tiveness  level  is  as  a  function  of  fleet  size.  A  campaign  scenario  should  include  mission-level 
insights,  as  well  as  broader  considerations,  such  as  basing  locations  of  RPAs  and  the  demand 
frequency  of  mission  occurrence  (i.e.,  ground  truth  on  target  distributions  in  time  and  space). 
PAF  has  developed  the  FSE  model  to  perform  this  campaign  analysis.  The  campaign  look 
afforded  by  FSE  results  in  a  required  force  size  under  varying  effectiveness  levels.  The  previ¬ 
ously  mentioned  individual  tools  and  the  mission-level  outcomes  from  SCOPEM  inform  the 
campaign  model,  FSE.  Last,  when  the  force  structure  evaluation  is  coupled  with  cost  analysis, 
a  cost-effectiveness  examination  of  the  candidate  systems  is  created. 
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Taken  together,  this  suite  of  models  and  tools  can  help  the  Air  Force  explore  the  most 
cost-effective  ways  to  take  advantage  of  the  unique  capabilities  of  RPAs  in  the  future.  PAF  is 
now  using  SCOPEM  to  study  a  set  of  roles  and  missions  for  next-generation  RPAs. 
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Abbreviations 


AAA 

antiaircraft  artillery 

ACC/A2 

Air  Combat  Command  Intelligence  Directorate 

ACC/A8 

Air  Combat  Command  Programs  and  Financial  Management  Directorate 

ADA 

air  defense  artillery 

AF/A2 

Air  Force  Intelligence  Directorate 

AFCCC 

Air  Force  Combat  Climatology  Center 

AFRL 

Air  Force  Research  Laboratory 

AI 

airborne  interceptor 

ANOPP 

Aircraft  Noise  Prediction  Program 

AOR 

area  of  responsibility 

BVR 

beyond  visual  range 

C2 

command  and  control 

C3 

command,  control,  and  communications 

CAP 

combat  air  patrol 

CAS 

close  air  support 

CEP 

circular  error  probable 

CFL 

cleared  flight  level 

CNR 

clutter-to-noise  ratio 

CONOPS 

concept  of  operations 

CPU 

central  processing  unit 

CRRA 

Capabilities  Review  and  Risk  Assessment 

dB 

decibel 

dBHz 

decibels  relative  to  1  hertz 
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dBJ 

decibels  relative  to  1  joule 

dBsm 

decibel  cross-section  relative  to  one  square  meter 

dBW 

decibel  watt 

DEAD 

destruction  of  enemy  air  defenses 

DoD 

U.S.  Department  of  Defense 

DTED 

digital  terrain  elevation  data 

ECM 

electronic  countermeasure 

ELINT 

electronic  intelligence 

EO 

electro-optical 

ERP 

effective  radiated  power 

ESAMS 

Enhanced  Surface-to-Air  Missile  Simulation 

EW 

early  warning 

FDOA 

frequency  difference  of  arrival 

FLOPS 

floating-point  operations  per  second 

FMV 

full-motion  video 

FOV 

field  of  view 

FSE 

Force  Structure  Effectiveness 

FY 

fiscal  year 

GBU 

guided  bomb  unit 

GCI 

ground  control  intercept 

GFLOPS 

one  billion  floating-point  operations  per  second 

GIQE 

general  image-quality  equation 

GMTI 

ground  moving  target  indicator 

GSD 

ground  sample  distance 

HQACC/A8 

Director  of  Requirements,  Headquarters  Air  Combat  Command 
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The  recent  employments  of  remotely  piloted  aircraft  (RPAs)  in  Operation  Iraqi  Freedom  and 
Operation  Enduring  Freedom  show  promising  expansion  possibilities  for  RPA  roles.  In  addi¬ 
tion,  near-term  sensor  technologies  offer  potential  for  new  applications  and  concepts  of  opera¬ 
tions  (CONOPS)  for  RPAs  within  the  next  five  years.  These  new  capabilities  will  allow  the  Air 
Force  to  collect  additional  information  on  an  area  or  target  of  interest,  which  then  could  be 
incorporated  into  missions  with  existing  systems  or  potentially  be  utilized  in  new  ways  that  are 
not  realizable  with  current  systems. 

Because  of  these  increased  RPA  capabilities  and  new  sensor  technologies,  the  environ¬ 
ment  is  ripe  to  consider  employment  of  RPAs  in  new  roles.  Thoughtful  study  into  these  pos¬ 
sibilities  is  prudent  as  the  Air  Force  explores  how  future  systems  and  CONOPS  can  help  fill 
capability  gaps. 

In  this  documented  briefing,  we  describe  a  suite  of  models  and  tools  developed  by  RAND 
Project  AIR  FORCE  (PAF)  that  can  contribute  to  this  effort.  All  of  the  models  developed  by 
PAF  were  built  within  the  past  five  years.  In  fiscal  year  (FY)  2010,  some  of  these  models  were 
improved,  and  additional  models  were  developed,  e.g.,  the  Force  Structure  Effectiveness  (FSE) 
model  used  to  determine  the  force  size  required  of  a  given  RPA  candidate  to  meet  mission 
objectives.  Existing  Air  Force  tools  were  integrated  into  RAND’s  suite  of  methodologies  as 
needed.  We  also  provide  examples  of  model  employment  for  evaluating  candidate  RPAs. 
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RPA  Evaluation  Methodology 


In  order  to  explore  the  mission  space  for  possible  future  RPA  roles,  a  systematic  analysis  is  fol¬ 
lowed  to  examine  the  trade-offs  between  the  various  candidate  systems.  This  slide  illustrates 
the  analytic  approach  developed  by  PAF.  The  analysis  of  potential  RPA  roles  begins  by  estab¬ 
lishing  the  capability  gaps  or  future  capabilities  desired  for  the  future  forces.  Candidate  plat¬ 
forms  must  then  be  defined,  and  subsystems  for  these  platforms  must  be  examined.  The  trade¬ 
offs  between  different  candidate  RPAs  or  manned  platforms  to  fill  a  particular  capability  gap 
are  explored  within  a  series  of  mission  vignettes.  The  vignettes,  once  selected,  are  modeled  to 
examine  the  different  candidate  systems  using  RAND’s  mission-level  analysis  model,  Systems 
and  CONOPS  Operational  Effectiveness  Model  (SCOPEM).  The  result  of  analyzing  the  can¬ 
didate  systems,  within  the  vignettes  under  varying  conditions,  is  an  effectiveness  matrix,  which 
shows  each  candidate’s  capabilities  for  a  selected  set  of  measures  of  effectiveness  (MOEs).1  To 
include  the  effects  of  conflict  time  in  the  analysis,  we  employ  an  FSE  model.  The  candidate 
systems  are  then  rated  by  an  appropriate  set  of  campaign-level  MOEs. 

Both  SCOPEM  and  the  FSE  model  are  described  in  more  detail  later  in  this  briefing. 


1  Measures  of  performance  (MOPs)  derived  from  SCOPEM  modules  include  detection  of  a  target  for  a  sensor,  line-of- 
sight  (LOS)  obscuration  from  terrain,  and  probability  of  kill  (Pk)  from  weapon  employment.  Bigger-picture  MOEs  result 
from  the  mission-level  analysis  from  SCOPEM,  e.g.,  the  candidate  platform  successfully  tracked  the  vehicle  for  30  minutes. 
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Mission  Vignettes  Are  Selected  to  Examine 
Gaps  in  Desired  Capabilities 


Potential  Sources  for 

Example  Vignettes 

Identifying  Capability  Gaps 

•  Detect  and  track  HVTs  in 

mountainous  and  urban  terrain 

•Air  Force  CRRA 

•  Conduct  SEAD/DEAD  in  a  major 

•  MSFD  scenarios 

combat  operation 

*  Discussions  with  salient  Air 

•  Detect  and  monitor  chemical- 

Force  offices 

weapon  manufacturing  sites  and 

personnel 

‘Combination  of  capabilities  needed  for  success  in  vignette 
•  Not  a  one-to-one  mapping 


‘Design  of  future  systems  and  vignettes  is  an  iterative  process 

*  Different  environmental  conditions 

*  Different  sizes  and  mix  of  friendly,  civilian,  and  enemy  forces 

*  Parameters  must  be  tuned  to  illuminate  differences 


To  explore  future  roles  and  missions  of  RPAs,  it  is  useful  to  determine  U.S.  Air  Force  capability 
gaps  that  could  potentially  be  met  by  RPA  platforms.  Vignettes  are  useful  to  shed  light  on  the 
mission  effectiveness  of  one  candidate  versus  another  under  varying  environmental  conditions. 

There  are  several  potential  sources  for  establishing  capability  gaps.  One  may  refer  to  the 
Air  Force  Capabilities  Review  and  Risk  Assessment  (CRRA)  to  include  in  the  analysis  the  Air 
Force’s  stated  capability  gaps.  Referencing  the  Multi-Service  Force  Deployment  (MSFD)  sce¬ 
narios  gives  a  joint  perspective  on  mission  needs  for  future  force  structure  planning.  Discus¬ 
sions  with  salient  Air  Force  offices  concerning  particular  mission  areas  are  another  source  for 
finding  capability  gaps. 

Once  the  desired  capabilities  are  established,  a  set  of  vignettes  is  selected  to  evaluate  the 
candidate  systems  within  these  situations  and  determine  their  mission  effectiveness.  Note  that 
it  often  takes  multiple  capabilities  to  successfully  complete  a  mission  within  a  vignette.  For 
example,  detecting  and  tracking  a  high-value  target  (HVT)  in  mountainous  or  urban  terrain 
require  the  capabilities  of  persistence,  observing  targets  in  challenging  terrain,  and  maintain¬ 
ing  track  of  individuals.  Conducting  suppression  of  enemy  air  defenses  (SEAD)  and  destruc¬ 
tion  of  enemy  air  defenses  (DEAD)  in  a  major  combat  operation  requires  detection  of  emitting 
radars,  high-resolution  imaging  and  accurate  geolocation  of  air  defenses,  penetrating  a  denied 
environment  and  surviving,  and  effectively  employing  weapons  to  destroy  the  target.  There¬ 
fore,  a  simple  one-to-one  mapping  between  capability  and  vignette  does  not  exist;  rather,  there 
is  a  more  complex  relationship  between  the  two. 

The  design  of  future  candidate  systems  and  vignettes  within  which  they  are  examined  is 
an  iterative  process.  Analysis  must  cover  different  environmental  conditions,  threats,  and  sizes 
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and  mixes  of  forces.  In  varying  these  parameters,  the  differences  among  candidate  systems  and 
CONOPS  are  illuminated. 
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RPA  Evaluation  Methodology 


Mission-level  analysis  provides  a  framework  within  which  candidate  systems  can  be  evaluated. 
Three  areas  of  analysis  fall  within  this  category:  candidate  platform  development  and  selec¬ 
tion,  subsystem  modeling,  and  simulations  of  systems  and  CONOPS  within  vignettes.  In  the 
following  slides,  we  discuss  the  tools  and  models  we  have  developed  that  contribute  to  this 
analysis  and  how  they  relate  to  one  another. 
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RAND’s  Systems  and  CONOPS  Operational 
Effectiveness  Model 


SCOPEM 


Platform 

•Orbit,  flight  path 
•Speed,  altitude 
•  CONOPS 


Sensors 

•NIIRS  capabilities  (EO,  IR,  SAR, 
FMV) 

•RCS  resolution  (ISAR,  MTI) 
•Sensitivity,  scan  cycle  (SIGINT) 

Weapons 
•Range  and  speed 
•Accuracy  (CEP) 


Targets 

•  NIIRS  characteristics  (EO,  IR,  SAR) 

•  RCS  (ISAR,  MTI) 

•  ERP,  shape,  frequency  (SIGINT) 


Environment 
•Terrain  (LOS) 

•Sea  state,  wind 
•Atmosphere 
•Space  weather 


Survivability 
•Air-to-air  threats 

•  Surface-to-air  threats 

•  Integrated  air  defenses 

Processing 

•Onboard  processing  (SWaP) 

•  Communications,  PED  delays 


There  are  many  factors  that  affect  operational  effectiveness  of  a  candidate  system;  in  this  docu¬ 
mented  briefing,  we  describe  the  tools  that  PAF  uses  to  evaluate  those  influences.  We  now 
present  our  mission-level  model,  which  integrates  these  factors  and  permits  analysis  of  mission- 
level  effectiveness. 

Once  mission-  or  vignette-level  analysis  is  complete,  the  analyst  has  an  understanding 
of  the  type  and  number  of  RPAs  that  are  needed  to  perform  a  particular  mission  within  a 
vignette.  But,  how  does  this  transfer  to  how  many  platforms  are  needed  for  a  force  size  of  that 
RPA  system?  A  higher-level  trade  analysis  model  could  be  used  to  explore  the  overall  force 
structure  requirements  and  associated  costs  of  a  given  candidate  system.  PAF  has  developed  a 
model,  the  FSE  model,  to  aid  in  this  analysis,  too. 
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SEAS/SCOPEM  Is  an  Agent-Based,  Time-Stepped, 
Stochastic  Modeling  Environment 


SEAS  is  maintained  and  developed  by  ExoAnalytic  Solutions 
for  SMC/XR.  SEAS  is  part  of  the  Air  Force  Standard  Analysis 
Toolkit  and  the  Air  Force  Space  Command  Modeling  and 
Simulation  Toolkit. 


An  agent  has  “autonomy” 


* 


Decisions 


SCOPEM,  originally  named  by  RAND  the  Collection  Operations  Model,  is  a  suite  of  inte¬ 
grated  modules  written  for  the  System  Effectiveness  Analysis  Simulation  (SEAS)  modeling 
environment.  SEAS  is  an  agent-based  representation  of  operations  that  proceeds  in  discrete 
time  steps,  usually  ranging  from  one  second  to  one  minute.  Each  agent  has  a  measure  of 
autonomy,  meaning  that  it  makes  decisions  based  on  its  perceptions,  as  dictated  by  the  sensors 
and  environment.2  RAND  has  added  a  large  number  of  new  properties  and  complex  deci¬ 
sion  rules  that  allow  SCOPEM  to  evaluate  operations  in  a  considerably  more  comprehensive 
manner  than  SEAS  alone  (e.g.,  detecting  a  target  with  one  sensor  and  cuing  a  second  sensor  or 
platform  to  track  or  attack  it).  Since  it  was  initiated  in  2005,  SCOPEM  has  supported  a  variety 
of  studies,  including  the  following: 


•  “Non-Traditional  Intelligence,  Surveillance,  and  Reconnaissance”  (FY  2008-2009,  Jody 
Jacobs,  Bart  Bennett;  sponsored  by  U.S.  Air  Force  Intelligence  Directorate  [USAF/A2]) 

•  “Satisfying  the  Demand  for  Surveillance  and  Reconnaissance  in  the  European  and  Afri¬ 
can  Theaters”  (FY  2008,  Carl  Rhodes;  sponsored  by  USAF/A2  and  U.S.  Air  Forces  in 
Europe  Intelligence  Directorate  [USAFE/A2]) 

•  “The  Role  of  Global  Hawk  in  Maritime  Surveillance”  (FY  2006—2007;  Sherrill  Lingel, 
Carl  Rhodes;  sponsored  by  Pacific  Air  Forces  Intelligence  Directorate  [PACAF/A2],  AF/ 
A2,  Air  Combat  Command  Intelligence  Directorate  [ACC/A2],  and  Air  Combat  Com¬ 
mand  Programs  and  Financial  Management  Directorate  [ACC/A8]) 


2  Satellites  are  represented  in  SCOPEM  as  modeled  by  the  government  in  SEAS. 
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•  “Tasking  and  Employing  USAF  Intelligence,  Surveillance,  and  Reconnaissance  Assets  to 
Support  Effects-Based  Operations”  (FY  2005-2006;  Sherrill  Lingel,  Carl  Rhodes;  spon¬ 
sored  by  PACAF/A2). 

Many  of  the  more-recent  improvements  to  SCOPEM  have  been  designed  to  allow  more- 
detailed  analysis  of  Air  Force  RPAs.  The  following  slides  discuss  how  SCOPEM  may  be  used 
to  support  future  work  in  this  area. 

Because  vignettes  may  have  very  different  measures  of  effectiveness  associated  with 
them — and  these  measures  may  be  refined  as  the  vignette  is  developed — the  simulation  should 
be  able  to  provide  flexible  output.  SCOPEM  provides  three  main  options. 

The  most  common  form  of  output  from  SCOPEM  is  a  report  of  every  detection  by  every 
sensor  on  every  time  step.  (Filters  can  be  applied  to  limit  attention  to  particular  sensor/target 
combinations  as  needed.)  Along  with  the  detection,  the  model  reports  perceived  target  param¬ 
eters,  such  as  location  and  velocity  (with  uncertainties),  and  reports  approximate  measures 
of  the  quality  of  the  detection  (e.g.,  National  Imagery  Interpretability  Rating  Scale  [NIIRS] 
rating,  signal-to-noise  ratio  [SNR]).  These  statistics  can  be  compiled  and  processed  afterward, 
e.g.,  in  a  spreadsheet,  to  generate  MOEs.  In  addition,  the  model  also  reports  any  targets  within 
range  that  were  not  detected.  This  allows  the  analyst  to  explore  which  factors  prevented  the 
detection.  Note  that  these  data  are  reported  for  the  output  only  and  are  not  used  within  the 
simulation.  Agents  are  never  permitted  to  peek  at  ground  truth. 

Another  option,  often  used,  is  the  ability  to  design  customized  output  of  any  parameter 
that  can  be  calculated  during  run  time  in  SCOPEM.  In  many  cases,  the  general  reporting 
above  is  used  until  the  particular  features  that  are  needed  can  be  identified,  at  which  point  a 
custom  routine  is  written  to  write  out  only  those  parameters. 

A  third  option  is  to  use  the  standard  output  files  of  SEAS  itself.  Because  SCOPEM  relies 
on  several  dozen  parameters  that  RAND  has  added  to  SEAS,  the  standard  output  file  does  not 
report  these.  However,  it  provides  basic  information  regarding  sensor  detections  and  weapon 
fire.  This  can  also  be  used  diagnostically. 
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Measuring  the  Operational  Effectiveness  of  a  Future 
System  Involves  Several  Fundamental  Challenges 

•  Effectiveness  can  be  measured  only  in  terms  of  outcomes  in  an 
operational  context 

-  Effectiveness  is  how  much  the  system  helps  the  warfighter  achieve  the 
operational  objective. 

-  Effectiveness  cannot  be  computed  from  technical  specifications  alone. 

-  A  large  trade  space  of  qualitatively  different  capabilities  exists. 

•  Whole  may  be  greater  or  lesser  than  sum  of  its  parts 

-  Must  be  evaluated  in  light  of  its  contribution  to  (or  reliance  on)  the 
capabilities  of  the  entire  force 

•  Future  systems  do  not  yet  exist 

-  CONOPS  for  using  system  may  be  new,  untested 

-  Simulations  depend  on  models  of  enemy  behavior 


The  operational  effectiveness  of  a  system  is,  in  essence,  the  degree  to  which  it  assists  the  war¬ 
fighter.  More  specifically,  it  can  be  defined  as  the  degree  to  which  the  system  improves  the 
warfighter’s  level  of  success  in  a  given  set  of  missions  or  enlarges  the  range  of  conditions  under 
which  the  warfighter  is  likely  to  be  successful  in  those  missions.  These  benefits  cannot  gener¬ 
ally  be  computed  from  the  technical  specifications  of  the  system  alone.  They  can  be  observed 
only  in  terms  of  outcomes  in  an  operational  context  that  includes  all  the  other  capabilities  in 
the  theater,  including  space  and  threat  systems.  Because  the  system  does  not  yet  exist,  placing 
the  system  in  operational  context  requires  constructive  simulation. 

Measuring  the  operational  effectiveness  of  a  future  system  therefore  entails  many  dif¬ 
ficulties.  First,  the  set  of  missions  over  which  the  system  will  be  judged — and  the  range  of 
environmental  conditions  considered — must  be  agreed  upon  by  the  stakeholders.  Otherwise, 
the  mission-level  analysis,  no  matter  how  sophisticated,  will  be  decidedly  less  useful  to  the 
decisionmaker.  Second,  the  system  cannot  be  fairly  judged  in  isolation.  It  must  be  evaluated  in 
light  of  its  contribution  to — or  reliance  on — the  capabilities  of  the  rest  of  the  force.  Moreover, 
many  systems  can  be  expected  to  provide  several  different  capabilities,  and  the  contributions 
of  one  might  depend  on  the  others.  The  set  of  missions  must  be  rich  enough  to  support  those 
interactions  without  becoming  so  complicated  that  the  effect  of  the  new  system  becomes  too 
subtle  to  discern.  Third,  CONOPS  for  using  the  new  system  might  be  new,  and  the  tactics 
might  be  untested  or  may  not  yet  exist.  Finally,  the  models  of  enemy  behavior  (on  which  the 
results  depend)  might  be  highly  speculative. 

These  difficulties  are  challenging  and  favor  certain  simulation  methods.  Clearly,  any 
model  used  needs  to  be  very  flexible,  able  to  simulate  different  “players”  in  a  wide  range  of  see- 
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narios  under  many  different  environment  conditions.  More  important,  however,  is  the  level  of 
resolution  of  the  model.  Unlike  engineering  simulations,  a  model  for  operational  effectiveness 
should  be  geared  toward  representing  many  features  at  lower  fidelity,  rather  than  a  few  features 
at  high  fidelity.3  This  is  driven  by  the  challenges  described  above. 

Simply  put,  there  is  much  uncertainty  in  simulating  operational  employment,  and  this 
should  be  acknowledged  at  the  outset.  That  the  final  results  could  depend  heavily  on  predic¬ 
tions  of  enemy  behavior  should,  by  itself,  be  enough  to  give  one  pause.  There  is  nothing  to  be 
gained  (and  much  to  be  wasted)  by  simulating  some  aspects  to  high  precision,  such  as  sensor 
systems  or  environmental  characteristics,  when  many  others  must  be  simplified.  We  must 
remember  that  the  purpose  of  this  modeling  effort  is  to  assist  decisionmaking.  In  some  situ¬ 
ations,  it  might  be  that  all  we  can  reliably  conclude  is  that  the  comparative  effectiveness  of 
two  systems  depends  strongly  on  factors  we  cannot  predict.  That  statement  has  more  value  to 
the  decisionmaker  than  a  highly  detailed  prediction  that  is  ultimately  based  on  questionable 
assumptions. 

The  ultimate  goal  of  operational  effectiveness  analysis  at  the  mission  level  is  to  allow 
the  analyst  to  trace  the  modeling  results  back  to  the  root  causes  of  success  or  failure  for  any 
given  alternative.  For  example,  was  there  an  environmental  condition  that  prevented  the  sensor 
from  detecting  a  target?  Or,  did  the  CONOPS  employed  limit  the  sensor’s  field  of  regard 
of  the  target  such  that  changing  the  orbit  location  would  bring  mission  success?  The  results 
must  be  accessible  to  the  analyst  and  explainable  to  the  decisionmaker  in  order  to  be  credible. 
SCOPEM,  along  with  the  associated  suite  of  tools,  provides  these  capabilities  to  the  analyst. 


3  An  engineering  simulation  can  model  subsystem  parts  with  high  fidelity  but  not  characterize  the  operational  impact  of 
those  systems. 
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RAND’s  Systems  and  CONOPS  Operational 
Effectiveness  Model 


SCOPEM 


Platform 

•Orbit,  flight  path 
•Speed,  altitude 
•  CONOPS 


Sensors 

•NIIRS  capabilities  (EO,  IR,  SAR, 
FMV) 

•RCS  resolution  (ISAR,  MTI) 
•Sensitivity,  scan  cycle  (SIGINT) 

Weapons 
•Range  and  speed 
•Accuracy  (CEP) 


Targets 

•  NIIRS  characteristics  (EO,  IR,  SAR) 

•  RCS  (ISAR,  MTI) 

•  ERP,  shape,  frequency  (SIGINT) 


Environment 
•Terrain  (LOS) 

•Sea  state,  wind 
•Atmosphere 
•Space  weather 


Survivability 
•Air-to-air  threats 

•  Surface-to-air  threats 

•  Integrated  air  defenses 

Processing 

•Onboard  processing  (SWaP) 

•  Communications,  PED  delays 


In  this  section,  we  take  a  closer  look  at  SCOPEM  and  discuss  individual  modules  in  more 
detail.  The  SEAS/SCOPEM  environment  includes  elements  of  the  platform,  environment, 
sensor,  targets,  and  processing  modules.  Survivability  and  weapon  employment  model  outputs 
provide  data  for  the  mission-level  modeling  in  SCOPEM. 
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In  FY  2010,  the  Analytic  Methods  Were  Integrated  for 
Future  Assessment  of  Candidate  RPAs 


SCOPEM 


Platform 

•Orbit,  flight  path 
•Speed,  altitude 

•CONOPS 


Sensors 

•NIIRS  capabilities  (EO,  IR,  SAR, 

•RCS  resolution  (ISAR,  MTI) 

nsitivity,  scan  c 

Weapons 
•Range  and  speed 
•Accuracy  (CEP) 


Targets 

■  NIIRS  characteristics  (EO,  IR,  SAR) 

•  RCS  (ISAR,  MTI) 

•  ERP,  shape,  frequency  (SIGINT) 


Environment 
•Terrain  (LOS) 

•Sea  state,  wind 
•Atmosphere 
•Space  weather 


Survivability 

•  Permissive,  contested  air 
defenses 

•Air-to-air  threats 


_ Processing _ 

•Onboard  processing  (SWaP) 

•Communications,  PED  delays 


The  suite  of  analytic  methods  described  above  and  detailed  in  the  following  pages  are  not 
limited  to  RPA  analysis  but  are  applicable  across  manned  airborne,  RPA,  space,  and  ground- 
based  systems.  Some  of  the  methods  make  use  of  existing  U.S.  Department  of  Defense  (DoD) 
resources,  such  as  the  aircraft  survivability  model  Enhanced  Surface-to-Air  Missile  Simulation 
(ESAMS)  or  the  Joint  Munitions  Effectiveness  Manual  (JMEM)  weapon  employment  model. 
Others  are  previously  developed  RAND  tools.  All  have  been  integrated  into  a  unique  suite  that 
is  designed  to  evaluate  RPAs  in  a  range  of  missions.  The  highlighted  portions  in  this  slide  rep¬ 
resent  the  methods  that  were  modified  and  developed  for  the  PAF  effort  in  FY  2010. 

The  following  slides  detail  individual  analytic  methods  in  the  areas  of  platform,  sensors, 
weapons,  targets,  environment,  survivability,  and  processing,  as  well  as  SCOPEM,  which  per¬ 
forms  mission-level  analysis,  and  the  FSE  model,  which  performs  campaign-level  analysis. 
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Analytic  Methods  Were  Developed  to  Examine  the 
Factors  That  Affect  Operational  Effectiveness 


SCOPEM 


Platform 

•Orbit,  flight  path 

•  Speed,  altitude 

•  CONOPS 


Sensors 

•NIIRS  capabilities  (EO,  IR,  SAR, 
FMV) 

•RCS  resolution  (ISAR,  MTI) 
•Sensitivity,  scan  cycle  (SIGINT) 

Weapons 
•Range  and  speed 
•Accuracy  (CEP) 


Targets 

■  NIIRS  characteristics  (EO,  IR,  SAR) 

•  RCS  (ISAR,  MTI) 

•  ERP,  shape,  frequency  (SIGINT) 


Environment 
•Terrain  (LOS) 

•Sea  state,  wind 
•Atmosphere 
•Space  weather 


Survivability 

•  Permissive,  contested  air 
defenses 

•Air-to-air  threats 


Processing 

•Onboard  processing  (SWaP) 
•Communications,  PED  delays 


We  apply  a  systematic  approach  to  defining  candidate  manned  and  unmanned  systems  and 
develop  appropriate  employment  concepts  for  each  tailored  to  the  mission  undertaken.  The 
next  set  of  slides  describes  our  approach  to  selecting  platforms  and  assessing  suitable  CONOPS 
to  be  evaluated  within  our  mission-level  analysis. 


14  Methodologies  for  Analyzing  Remotely  Piloted  Aircraft  in  Future  Roles  and  Missions 


First  Candidate  Platforms  and  CONOPS  for 
Employing  Them  Are  Defined 

•  Developing  preliminary  RPA  performance  specifications  is  an  iterative  process 

•  May  include  trade  analyses  between  platform  and  payload  requirements 

•  The  parameters  considered  in  the  RPA  performance  depend  on  the  mission  and 
may  include  the  following: 

•  Platform:  Dash  and  loiter  speed,  maximum  altitude,  endurance, 
maneuverability,  survivability  (e.g.,  signature,  active  defenses),  size  of 
platform,  SWaP  capacity,  processing  and  communications  capabilities 

•  Payload:  SWaP  requirements,  active/passive,  detection  range,  resolution, 

FOV,  measurement  quality  (e.g.,  NIIRS),  agility,  processing  capability, 
communications  requirements 


We  first  define  candidate  RPA  or  manned  platforms  and  work  with  the  Air  Force  to  develop 
CONOPS  for  employing  them.  Based  on  the  mission  objectives,  the  platform  constraints, 
and  the  terrain  and  threat  environments,  we  develop  preliminary  platform  performance  speci¬ 
fications.  Establishing  specifications  is  an  iterative  process  that  may  include  a  trade  analysis 
between  the  platform  and  the  payload  requirements.  The  first  iteration  begins  with  the  opera¬ 
tional  need.  From  that  need,  we  work  with  subject-matter  experts  to  develop  a  first  set  of  plat¬ 
form  characteristics. 

The  RPA  platform  characteristics  under  consideration  may  include  dash  and  loiter  speed, 
maximum  altitude,  endurance,  maneuverability,  survivability  (e.g.,  signature  reduction  and 
active  defenses),  payload  size,  weight,  and  power  (SWaP)  capacity,  and  processing  and  com¬ 
munications  capabilities. 

Payload  characteristics  include  SWaP  and  communications  requirements.  Additional 
characteristics  for  sensor  payloads  may  include  active  or  passive  sensors  or  both,  detection 
range,  resolution,  held  of  view  (FOV),  measurement  quality  (NIIRS),  agility,  and  processing 
requirements. 

Sensor  performance  is  analyzed  separately,  as  discussed  later  in  this  documented  briefing. 
If  sensor  performance  results  fail  to  meet  mission  requirements,  then  an  iterative  process  in 
which  the  system  architecture  is  refined  and  reevaluated  is  conducted  to  obtain  an  acceptable 
initial  architecture. 

Trades  between  the  number  of  RPAs  and  their  platform  and  payload  performance  param¬ 
eters  are  also  included  in  the  approach,  as  is  discussed  in  this  document.  The  process  will 
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include  sensitivity  analysis  to  evaluate  the  impact  of  performance  requirements,  related  tech¬ 
nology  maturity,  and  risks  (and  cost)  on  overall  mission  performance. 
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PAF’s  Tracking  Estimation  Tool  Helps 
Identify  Orbits  for  Surveillance 


MATLAB-Based  Tracking  Estimation  Tool 

Inputs  Motion  simulator 


We  use  the  tool  to  identify  the  best  orbits  for  tracking  targets 
and  implement  these  orbits  in  SCOPEM. 


An  appropriate  CONOPS  for  each  mission  must  be  developed  along  with  each  candidate 
RPA  or  manned  platform.  Sources  for  candidate  CONOPS  include  warfighter  tactics,  tech¬ 
niques,  and  procedure  documents  and  direct  warfighter  input.  For  intelligence,  surveillance, 
and  reconnaissance  (ISR)  missions,  an  important  aspect  of  this  CONOPS  is  the  flight  pattern 
or  orbit  for  the  platform.  Our  tracking  estimation  tool  helps  identify  orbits  for  surveillance 
missions  by  providing  MOEs.  Increased  average  time  for  which  a  target  is  tracked  and  maxi¬ 
mum  number  of  targets  that  can  be  tracked  are  two  examples  of  MOEs.  Flight  paths  are  also 
important  for  survivability  concerns  in  contested  environments  but  are  not  considered  within 
this  tool;  instead,  they  are  considered  in  the  larger  mission-effectiveness  model. 

The  tracking  estimation  tool  consists  of  three  interconnected  models,  as  shown  in  the 
slide.  The  primary  inputs  to  the  tool  include  information  about  the  sensor  platform  orbit  and 
target  geometry,  the  average  spacing  of  the  targets  with  respect  to  each  other,  the  sensor  revisit 
rate  (how  often  the  sensor  sweeps  over  the  target),  and  the  sensor  field- of-regard  specifications. 
These  inputs  feed  into  a  motion  simulator  model  that  updates  the  position  of  the  sensor  plat¬ 
form  along  its  orbit  and  the  position  of  the  target  along  its  path  during  a  sequence  of  time  steps. 
The  output  is  then  fed  into  the  hypothesis  model,  which  estimates  the  probability  that  track  of 
the  target  will  be  maintained  between  subsequent  revisits  of  the  sensor.  It  does  this  by  evaluat¬ 
ing  the  probability  that  a  position  measurement  of  another  target  may  be  confused  with  that 
of  the  target.  This  probability  is  then  passed  to  the  Bernoulli  model,  which  accumulates  the 
outputs  of  the  hypothesis  model  to  determine  the  aggregate  probability  distribution  of  main¬ 
taining  track  as  a  function  of  total  elapsed  time  since  the  track  was  established. 
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The  tracking  estimation  tool  can  be  used  to  identify  the  best  surveillance  orbits  for  RPAs 
to  use  during  tracking  missions.  These  orbits  are  incorporated  into  simulation  tools,  such  as 


SCOPEM. 
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How  Accurately  RPAs  Can  Geolocate  Targets 
Using  SIGINT  Is  Estimated 

•  Tool  estimates  the  accuracy 
of  cooperative  SIGINT 
geolocation 

•  It  implements  a  TDOA/ 

FDOA  geolocation  algorithm 
in  a  MATLAB-based 
simulation  tool 

•  Inputs: 

-  RPA,  target  range,  and 
geometry 

-  Signal  characteristics 

-  Navigation  precision 

•  Output: 

-  Geolocation  accuracy 
error  ellipse 


Another  factor  that  may  influence  CONOPS  and  sensor  selection  for  ISR  missions  is  a  candi¬ 
date  platform’s  ability  to  geolocate  a  target  from  its  electronic  emissions.  We  developed  a  tool 
to  estimate  how  accurately  candidate  platforms  can  geolocate  an  emitting  target.  Although 
some  platforms  can  perform  direction  finding  themselves,  the  accuracy  is  poor,  and  multiple 
platforms  operating  together  would  improve  accuracy.  For  example,  two  platforms  will  receive 
these  electronic  emissions  at  slightly  different  times  (assuming  they  are  at  different  ranges  from 
the  target).  Hence,  there  will  be  a  time  difference  of  arrival  (TDOA).  Similarly,  there  will  be  a 
frequency  difference  of  arrival  (FDOA)  due  to  Doppler  shift  created  by  the  motion  of  the  plat¬ 
forms.  The  source  of  these  electronic  emissions  can  be  located  as  the  intersection  of  TDOA  and 
FDOA  contours  on  the  earth’s  surface.  We  implemented  this  TDOA/FDOA  geolocation  tech¬ 
nique  in  a  MATLAB  simulation  tool.  Inputs  to  the  tool  include  platform  and  target  position 
and  velocity,  characteristics  of  the  emitted  signals  and  receiving  electronics  on  the  platforms, 
and  accuracy  of  the  platform  navigation  systems.  Inferred  in  this  analysis  are  the  platforms’ 
flight  paths  relative  to  the  target.  The  output  of  the  tool  is  the  predicted  geolocation  accuracy. 
The  tool  can  be  used  to  determine  platform  sensor  requirements  for  targeting,  suitable  flight 
paths,  and  other  mission  tasks. 

Geolocating  targets  based  on  signals  intelligence  (SIGINT)  collections  can  be  used  as  a 
means  of  cuing  additional  surveillance  sensors  on  the  target.  For  example,  within  SCOPEM, 
when  a  target  is  geolocated  with  TDOA  or  FDOA,  an  area  on  the  ground  is  provided  (the  geo¬ 
location  error  ellipse)  of  where  the  target  could  be  located  and  where  we  may  want  to  collect 
imagery  as  a  directed  search.  Electro-optical/infrared  (EO/IR)  or  full-motion  video  (FMV) 


Target:  insurgent  operating 
a  hand-held  radio 
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sensors  may  then  be  tasked  to  image  the  area  to  identify  the  target  and  provide  a  more  accurate 
location. 
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Analytic  Methods  Were  Developed  to  Examine  the 
Factors  That  Affect  Operational  Effectiveness 


SCOPEM 


Platform 

•Orbit,  flight  path 
•Speed,  altitude 
•CONOPS 


Sensors 

•NIIRS  capabilities  (EO,  IR,  SAR, 
FMV) 

•RCS  resolution  (ISAR,  MTI) 
•Sensitivity,  scan  cycle  (SIGINT) 

Weapons 
•Range  and  speed 
•Accuracy  (CEP) 


Targets 

•  NIIRS  characteristics  (EO,  IR,  SAR) 

•  RCS  (ISAR,  MTI) 

•  ERP,  shape,  frequency  (SIGINT) 


Environment 
•Terrain  (LOS) 

•Sea  state,  wind 
•Atmosphere 
•Space  weather 


Survivability 

•  Permissive,  contested  air 
defenses 

•Air-to-air  threats 


Processing 

•Onboard  processing  (SWaP) 
•Communications,  PED  delays 


We  have  developed  a  suite  of  models  to  calculate  MOEs  for  many  types  of  ISR  sensors.  These 
ISR  sensors  may  reside  on  RPAs,  manned  platforms,  or  satellites.  Together,  these  provide 
SCOPEM  with  a  means  of  evaluating  how  specific  platforms  perform  ISR  collections  against 
specific  targets  under  various  environmental  conditions.  The  following  slides  describe  our 
methodologies  for  analyzing  EO/IR,  FMV,  wide-area  airborne  surveillance  (WAAS),  ground 
moving  target  indicator  (GMTI),  synthetic  aperture  radar  (SAR),  and  electronic  intelligence 
(ELINT)  sensor  performance. 
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EO/IR  NIIRS  as  a  Function  of  Range  Is  Determined 
from  Contractor  Data  and  the  GIQE 

•  A  contractor  or  program  office  may  supply  NIIRS  as  a  function  of  range  for 
sensors  already  fielded  or  under  development. 

*  When  NIIRS  data  are  not  available,  the  GIQE  is  used  to  estimate  performance. 


NIIRS w  =  10.25 1  -  a  log10  (GSDcm  )+b\ogm(RERGM  )-0.656 H- 

0.344G 

SNR 

GSDGm 

:  GSD  based  on  sensor  FOV,  platform  orientation,  and  range 

SNR 

:  SNR  based  on  target  type,  atmosphere,  weather,  and  internal  sensor  noise 

RERGm 

:  Normalized  RER  based  on  pixel  density 

H,  G 

:  Respectively,  mean-height  overshoot  and  noise  gain,  measures  of  the  effect  of  MTF 
compensation 

*  SCOPEM  uses  NIIRS  data  to  calculate  MOEs 

-  NIIRS  performance  curves  for  all  EO/IR  sensors  are  provided  to  SCOPEM. 

-  Military  and  civilian  NIIRS  tables  define  the  NIIRS  level  required  to  detect,  classify,  or 
identify  a  target. 

-  SCOPEM  determines  the  probability  of  detecting,  classifying,  or  identifying  targets  in 
diverse  scenarios. 


For  ISR-related  capabilities,  one  MOE  may  be  a  sensor’s  ability  to  detect  or  identify  a  target. 
The  MOEs  for  EO  and  IR  sensors  are  defined  as  the  probabilities  of  detecting,  classifying,  and 
identifying  objects  of  interest  in  varying  scenarios. 

The  ability  of  an  EO/IR  sensor  to  detect  and  identify  a  target  may  be  quantified  using  the 
NIIRS.  NIIRS  level  is  a  measure  of  the  interpretable  content  of  an  image,  based  on  the  empiri¬ 
cal  performance  of  experienced  analysts.  Table  1  shows  the  NIIRS  level  necessary  to  detect  and 


Table  1 

National  Imagery  Interpretability  Rating  Scale  Necessary  to  Detect  or  Identify  Notional  Targets 


EO  NIIRS 

IR  NIIRS 

Target 

Detect 

Identify 

Detect 

Identify 

Group  of  trucks 

4 

6 

4 

6 

Truck 

4 

6 

4 

6 

Motorcycle 

5 

6 

5 

7 

Group  of  people 

6 

8 

7 

8 

Person 

6 

8 

7 

9 

Person  with  weapon 

7 

8 

8 

9 

SOURCE:  Imagery  Resolution  Assessments  and  Reporting  Standards  Committee,  1996. 
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identify  various  target  types.  These  estimates  are  a  subset  of  much  larger  military  and  civilian 
NIIRS  tables  developed  by  the  U.S.  government. 

The  NIIRS  approach  accounts  for  many  factors  that  affect  image  interpretability,  includ¬ 
ing  sensor  resolution,  image  sharpness,  noise,  contrast,  and  human  effectiveness  at  interpreting 
the  image.  However,  NIIRS  does  not  include  effects  of  foliage,  camouflage,  clouds,  or  other 
objects  that  may  obstruct  the  view  of  a  sensor. 

NIIRS  level,  as  a  function  of  range,  is  often  obtained  via  hardware  testing  by  sensor  con¬ 
tractors.  However,  NIIRS  level  may  also  be  estimated  using  the  general  image-quality  equation 
(GIQE).4  The  GIQE  is  a  model  to  predict  NIIRS  level  based  on  sensor,  target,  and  environ¬ 
mental  characteristics  (Leachtenauer  and  Driggers,  2001,  p.  301).  Target  effects  (aspect,  size, 
contrast)  are  captured  in  the  ground  sample  distance  (GSD)  term  and,  potentially,  the  SNR 
ratio  term.  Atmospheric  effects  and  weather  are  included  via  the  SNR  term.  Sensor  character¬ 
istics  define  the  GSD  and  modulation  transfer  function  (MTF)-related  (relative  edge  response 
[RER]),  and  to  a  lesser  degree,  H  and  G  terms.  This  version  of  the  GIQE  assumes  that  hard¬ 
copy  imagery  is  being  exploited.  Also,  any  degradation  due  to  image  compression  is  ignored. 

NIIRS  performance  results  are  used  to  examine  the  performance  of  a  given  sensor  to  col¬ 
lect  on  a  particular  target.  When  evaluating  the  candidate  platform  performance  in  a  vignette, 
SCOPEM  must  test,  at  each  time  step  over  the  duration  of  a  mission,  whether  a  platform  (with 
a  defined  EO/IR  sensor  package)  is  able  to  attain  an  adequate  NIIRS  level  to  detect,  classify,  or 
identify  a  target.  Using  Monte  Carlo  analysis,  SCOPEM  provides  the  probability  of  detecting, 
classifying,  and  identifying  a  target,  as  well  as  the  EO/IR  sensor’s  impact  on  overall  mission 
success. 


4  In  the  GIQE,  a  and  b  are  constants;  when  defined,  a  =  3.32  if  RER  >  0.9  or  3.16  if  RER  <  0.9,  and  b  =  1.559  if  RER  >  0.9 
or  2.817  if  RER  <  0.9. 
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Performance  of  Motion  Video  and  WAAS  Sensors 
is  Modeled  by  Leveraging  NIIRS  Analysis 

•  EO/IR  GIQE  and  NIIRS  tables  for  still  imagery  are  used  to 
qualitatively  approximate  the  intelligence  value  of  motion  imagery 

-  VNIIRS  has  recently  been  accepted  by  the  Motion  Imagery 
Standards  Board 

-  However,  developing  the  associated  MIQE  is  still  an  active  area 
of  research  (i.e.,  one  has  not  been  fully  developed  and  accepted) 

•  To  simulate  motion  imagery,  images  are  taken  contiguously — at  each 
time  step — in  SCOPEM 

-  This  results  in  the  achievable  NIIRS  level  as  a  function  of  time 
for  each  sensor  in  each  scenario 

-  NIIRS  level  as  a  function  of  time  is  used  to  determine  probability 
of  detection,  classification,  identification,  and  maintaining  track 

•  SCOPEM  includes  initial  Gorgon  Stare  WAAS  CONOPS 


The  EO/IR  GIQE  and  NIIRS  tables  for  still  imagery  are  used  to  approximate  the  interpret- 
ability  of  motion  imagery.  Although  a  video  NIIRS  (VNIIRS)  has  recently  been  accepted  by 
the  National  Geospatial-Intelligence  Agency  (NGA)  as  a  subjective  quality  scale  for  rating  the 
intelligence  value  of  airborne  motion  imagery  in  the  visible  spectrum,  developing  the  associated 
motion  imagery-quality  equation  (MIQE)  is  still  an  area  of  active  research  (Motion  Imagery 
Standards  Board,  2009;  Young  and  Bakir,  2009).  Because  the  MIQE  is  not  yet  verified,  the 
GIQE  and  still-imagery  NIIRS  are  used  to  evaluate  FMV  sensor  performance  in  SCOPEM. 

Motion  imagery  is  simulated  as  a  series  of  still  images.  This  results  in  the  achievable 
NIIRS  level  as  a  function  of  time  for  each  EO/IR  FMV  sensor  in  each  scenario.  NIIRS  level 
as  a  function  of  time  may  be  used  to  determine  the  probability  of  detecting,  classifying,  and 
identifying  a  target  in  the  same  way  as  previously  described  for  still  imagery. 

SCOPEM  also  simulates  an  EO/IR  FMV  sensor’s  ability  to  track  a  target  of  interest.  The 
ability  to  track  a  target  with  FMV  will  be  a  function  of  the  sensor’s  ability  to  maintain  a  target 
in  view,  at  a  specified  NIIRS  level.  In  environments  without  traffic,  the  sensor  must  maintain 
a  NIIRS  level  above  that  required  to  detect  the  object.  In  high-traffic  scenarios,  the  sensor  must 
have  a  NIIRS  level  above  that  required  to  identify  the  target  in  order  to  distinguish  it  from 
nearby  “confusers.”  The  sensor  can  lose  track  of  the  target  for  periods  of  time  during  a  scenario. 
Thus,  the  probability  of  track  is  defined  as  the  average  percentage  of  time  that  a  target  is  both 
within  the  sensor  FOV  and  at  the  required  NIIRS  level. 

One  example  of  an  EO/IR  FMV  capability  currently  modeled  in  the  SCOPEM  environ¬ 
ment  is  the  WAAS  Gorgon  Stare.  In  the  current  design,  Gorgon  Stare  contains  five  black-and- 
white  EO  cameras,  as  well  as  four  IR  cameras.  A  relatively  seamless  view  is  produced  from 
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each  of  the  cameras  (within  each  bandwidth)  via  stitching  algorithms;  this  provides  approxi¬ 
mately  12  square  kilometers  of  coverage  on  the  ground  in  the  sensor  FOY  at  any  time.  Current 
Gorgon  Stare  CONOPS  require  the  associated  platform  to  stay  at  a  constant  altitude  during 
sensor  operations,  implying  a  constant  attainable  maximum  NIIRS  level  (Prociuk,  2009). 
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The  GMTI/Tracking  Model  Allows  SCOPEM  to  Compute 
Probabilities  of  Detection  and  Maintaining  Track 


*  PD  of  moving  targets 

-  Based  on  physical  radar,  target,  and  clutter 
characteristics 

-  Predicts  signal  and  clutter  energy  received  by  the 
radar 

*  Probability  of  maintaining  track 

*  Revisit  interval  incorporates  sensor,  target,  and 
environmental  characteristics 

-  Based  on  a  model  by  Mori,  Chang,  and  Chong* 

-  The  GMTI  detection  model  results  in  estimates  of 
target  SCNR  and  PD 

-  The  tracking  model  results  in  the  probability  of 
maintaining  track  on  a  target  of  a  given  SCNR  within 
SCOPEM 


*  Mori,  Chang,  and  Chong,  1992. 


Maintaining  track  of  a  mobile  target  is  important  for  many  ISR  missions.  PAF  developed  three 
tools  to  address  different  aspects  of  tracking.  Earlier,  we  presented  our  tracking  estimation  tool 
that  helps  identify  suitable  orbits  based  on  the  vehicle  traffic  density,  the  target’s  motion  uncer¬ 
tainties,  and  the  geometry  between  the  sensing  platform  and  the  target.  We  also  discussed 
tracking  with  FMV,  in  which  the  “tracking”  uses  the  human  eye  or  analysis  software.  Tracking 
moving  targets  using  GMTI  radar  is  another  way  to  track  targets,  and  PAF’s  GMTI/tracking 
model  is  our  third  model  related  to  tracking.  This  model  computes  the  probability  of  detec¬ 
tion  (PD)  and  maintaining  track  of  a  target  and  includes  the  effects  of  background  clutter  on 
detecting  a  moving  target. 

Under  appropriate  simplifying  assumptions,  PD  is  related  to  signal-to-clutter-plus-noise 
ratio  (SCNR)  and  the  probability  of  a  false  alarm  (PFA)  by  a  simple  expression.  The  inset  to 
this  slide  shows  PD  versus  slant  range. 

The  probability  of  maintaining  track  versus  the  revisit  interval  incorporates  sensor,  target, 
and  environmental  characteristics.  The  model  has  been  implemented  in  SCOPEM  as  tables 
of  probabilities  of  maintaining  track  for  a  range  of  revisit  intervals,  target  orientations,  and 
SCNRs  at  a  set  of  reference  values  that  includes  the  target  density  at  each  time  step.  Details  of 
the  GMTI  model  implementation  are  described  in  “Additional  Detail  on  Selected  Models”  at 
the  end  of  this  briefing. 
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The  SAR  Model  Predicts  Detection  or  Classification 
Based  on  Exceeding  Threshold  NIIRS  Values 

•  The  SAR  model  uses  the  SAR  GIQE  for  NIIRS 
but  modified  to  use  an  “effective”  resolution 
versus  grazing  angle  that  depends  on  target 
dimensions  and  accounts  for  the  effects  of 

-  Differing  resolution  on  horizontal  and  vertical 
surfaces 

-  Grazing  dependence  of  RCS  of  corner 
reflectors 

-  Shadowing 

a  SCOPEM  uses  this  model  to  predict  the 
potential  to  detect  or  classify  targets  by  SAR 

-  NIIRS  thresholds  are  specific  to  target  types 

quality  at  very  high  and  low 
grazing  angles 


The  SAR  model  predicts  detection  or  classification  based  on  exceeding  threshold  NIIRS  levels. 
A  NIIRS  has  been  adapted  for  application  to  SAR  images.  This  measure  emphasizes  the  impor¬ 
tance  of  GSD  on  image  quality.  NIIRS  is  a  useful  measure  because  of  its  familiarity  to  intel¬ 
ligence  analysts  but  also  because  performance  requirements  for  sensors  are  sometimes  stated  in 
terms  of  NIIRS.  Although  the  GIQE  is  better  established  for  EO/IR  than  for  SAR,  Driggers 
et  al.  have  attempted  to  calibrate  a  relationship  between  SAR  NIIRS  and  IR  NIIRS,  providing 
a  means  for  computing  SAR  NIIRS  as  a  function  of  GSD  (Driggers  et  al.,  2003). 

The  Driggers  et  al.  model,  like  NIIRS  models  for  other  sensors,  does  not  explicitly  capture 
dependencies  on  target  or  environmental  parameters,  nor  details  of  sensors  other  than  its  GSD. 
For  example,  it  does  not  take  into  account  the  target’s  aspect  to  the  radar,  clutter  statistics, 
or  the  asymmetry  of  range/azimuth  resolution  cells.  It  does  not  make  a  direct  connection  to 
detection-theoretic  concepts,  such  as  PD  and  PFA.  With  some  ingenuity,  it  may  be  possible  for 
the  analyst  to  account  for  some  of  these  effects  through  the  degrees  of  freedom  in  the  GIQE. 

Prior  to  the  FY  2010  effort,  we  modeled  SAR  NIIRS  using  expressions  calibrated  to  spe¬ 
cific  radars,  with  calibration  often  over  a  limited  domain  of  grazing  angles.  This  model  did 
not  readily  allow  generalization  to  systems  other  than  those  for  which  it  was  calibrated,  and  it 
imposed  a  sharp  cutoff  such  that  SAR  provided  no  useful  information  at  grazing  angles  below 
8  degrees  to  account  for  expected  degradation  in  image  quality.  In  the  present  effort,  we  devel¬ 
oped  the  SAR  NIIRS  model  for  application  to  radars  for  which  we  lack  calibration  data  and 
implemented  a  signal  model  that  provides  graceful  degradation  of  image  quality  at  low  grazing 
angles,  without  imposing  a  somewhat  ad  hoc  and  sudden  cutoff. 


NIIRS  versus  slant  range  (km) 


Example  shows  loss  of  Image 
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To  model  radars  that  lack  calibration,  we  implemented  the  Driggers  et  ah  expression  that 
relates  SAR  NIIRS  to  IR  NIIRS.  This  computation  depends  mainly  on  the  SAR  resolution  and 
does  not  depend  on  the  other  factors  in  the  GIQE  that  are  usually  of  secondary  importance  in 
imaging  sensors  (such  as  the  RER  and  height  overshoot  terms).  By  replacing  the  GSD  with  an 
“effective  resolution”  that  takes  into  account  target  features  and  viewing  geometry,  we  achieve 
a  graceful  degradation  in  performance  at  low  grazing  angles,  which  we  explain  in  “Additional 
Detail  on  Selected  Models”  at  the  end  of  this  briefing. 

This  model  was  implemented  in  SCOPEM  in  the  form  of  a  look-up  table  that  gives 
NIIRS  as  a  function  of  the  radar  resolution  in  the  slant  plane  and  grazing  angle. 
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ELINT  Model  Represents  Receiver/Emitter 
Properties  and  the  Detection  Processes 

•  Receiver  sensitivity  derives  from  basic  physics  of  signal  detection 

sensitivity  =  antenna  gain  +  threshold  —  radome  loss  —  noise  figure 
+  k  T0  +  bandwidth  —  1 0  log10  (  cos3  W), 

where  sensitivity  is  measured  in  decibel  watts  (dBW),  k7^  is  measured  in  decibels  relative  to  1  joule  (dBJ), 
and  bandwidth  is  measured  in  decibels  relative  to  1  hertz  (dBHz). 

•  Emitter  /  receiver  characteristics,  plus  transmission  factors, 
determine  PD  and  geolocation  accuracy 


Emitter  characteristics  Receiver  characteristics 


The  most  complex  single  module  within  SCOPEM  is  the  ELINT  detection  model.  In  keeping 
with  the  broad  approach  used  for  other  sensor  models,  we  do  not  simulate  details  of  the  propa¬ 
gation  of  electromagnetic  waves.  Rather,  the  model  characterizes  certain  average  properties  of 
the  target  (emitter)  and  the  sensor  (receiver) — in  this  case,  frequency  and  power — to  deter¬ 
mine  the  likelihood  of  detection  using  standard  physics  and  related  environmental  factors. 

Emitters  are  characterized  by  average  frequency  (middle  of  band)  and  effective  radiated 
power  (ERP).  The  ERP  is  allowed  to  take  different  values  in  different  directions  to  represent 
main  lobes,  side  lobes,  and  back  lobes.  The  pattern  of  emission  may  be  set,  and  the  emitter  is 
allowed  to  rotate.  These  characteristics  are  matched  by  receiver  sensitivity  (decibel  watt  [dBW]) 
across  various  frequency  bands  and  signal  search  patterns,  including  field  of  regard,  beam- 
width,  scanning  bandwidth,  and  the  scanning  cycle — which  may  emphasize  some  frequency 
bands  over  others.  To  detect  the  emitter,  the  receiver  must  have  sufficient  sensitivity  and  must 
be  looking  at  the  right  frequency  band  (accounting  for  Doppler  shifts),  in  the  right  direction, 
at  the  right  time. 

Attenuation  of  the  signal  as  it  travels  from  the  emitter  to  the  receiver  is  a  straightforward 
function  of  distance,  reflecting  the  inverse  square  law.  However,  other  signals  at  the  same  fre¬ 
quency  may  interfere.  All  signals  received  on  the  same  frequency  at  the  same  time  are  added 
(incoherently),  and  the  primary  signal  must  exceed  the  sum  of  the  others  by  a  given  threshold, 
e.g.,  6  dB.  Signals  may  use  communications  protocols  to  divide  the  spectrum  by  frequency  or 
time;  this  may  be  used  to  prevent  interference.  Receiver  sensitivity  also  falls  off  as  a  function  of 
reception  angle  relative  to  the  boresight. 
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The  geolocation  accuracy  of  the  emitter  is  determined  by  direction-finding  and  ranging 
capabilities,  which  may  depend  on  relative  angles  and  the  received  signal  strength.  An  aircraft 
may  run  a  line  of  bases,  that  is,  receive  the  signal  from  many  points  along  a  flight  path  in  order 
to  triangulate  its  location  more  precisely.  For  simplicity,  we  use  a  planar  earth  for  these  calcu¬ 
lations.  Fusing  the  error  ellipses  is  equivalent  to  multiplying  the  Gaussian  functions  that  give 
rise  to  them. 

With  this  model,  we  can  represent  many  of  the  relevant  engineering  characteristics  of  the 
emitter  that  are  expected  to  give  rise  to  its  performance.  We  can  also  represent  some  simple 
CONOPS  for  emitting  signals  and  searching  for  them.  When  the  factors  that  give  rise  to 
receiver  sensitivity,  such  as  the  radome  loss,  are  not  available,  an  empirical  measure  of  the  sen¬ 
sitivity  may  be  used. 
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Analytic  Methods  Were  Developed  to  Examine  the 
Factors  That  Affect  Operational  Effectiveness 


SCOPEM 


Platform 

•Orbit,  flight  path 
•Speed,  altitude 
•CONOPS 


Sensors 

•NIIRS  capabilities  (EO,  IR,  SAR, 
FMV) 

•RCS  resolution  (ISAR,  MTI) 
•Sensitivity,  scan  cycle  (SIGINT) 

Weapons 
•Range  and  speed 
•Accuracy  (CEP) 


Targets 

■  NIIRS  characteristics  (EO,  IR,  SAR) 

•  RCS  (ISAR,  MTI) 

•  ERP,  shape,  frequency  (SIGINT) 


Environment 
•Terrain  (LOS) 

•Sea  state,  wind 
•Atmosphere 
•Space  weather 


Survivability 

•  Permissive,  contested  air 
defenses 

•Air-to-air  threats 


Processing 

•Onboard  processing  (SWaP) 
•Communications,  PED  delays 


An  important  factor  in  assessing  candidate  RPAs  is  the  employment  of  weapons,  whether 
by  RPAs  in  a  mission  or  by  enemy  surface-to-air  missiles  (SAMs)  against  RPAs,  such  as  in  a 
denied  environment.  PAF’s  approach  to  address  weapon  employment  leverages  existing  Air 
Force  and  RAND  data  sources  and  tools  and  provides  SCOPEM  with  a  means  of  evaluating 
weapon  effect  on  mission  outcome.  The  next  slide  describes  our  weapon  employment  analysis. 
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Model  Exists  Within  SCOPEM  for  RPA  Weapon 

Employment 


The  weapon  model  in  SCOPEM  is  a  natural  extension  of  the  sensor  model.  In  order  for  an 
engagement  to  occur,  a  sensor  associated  with  a  given  platform  must  have  a  track  or  location 
and  identification  of  the  target  (depending  on  the  rules  of  engagement).  Sensors  to  do  this  may 
be  carried  either  by  the  platform  or  by  the  weapon  itself.  The  model  is  agnostic  to  what  the 
firing  agents  or  targets  are,  and  it  can  be  employed  with  a  candidate  platform,  such  as  an  RPA 
against  a  fixed  ground  target  or  against  a  mobile  SAM.  The  sensor  model  outputs  whether  the 
sensor  was  able  to  provide  the  necessary  information  (e.g.,  NIIRS  level  to  identify  the  target) 
in  order  to  engage  the  target,  which  is  fed  into  the  rules  of  engagement  for  any  given  scenario. 
If  the  sensor  data  are  of  sufficient  quality  to  justify  engagement  and  the  rules  of  engagement 
are  met,  the  weapon  model  then  begins  an  engagement  sequence. 

Pk  and  circular  error  probable  (CEP)  against  particular  hardened  and  nonhardened  tar¬ 
gets  are  coded  into  the  weapon  model  for  a  variety  of  weapons  from  the  JMEM  data  and 
empirical  reports  from  the  Air  Force  Weapons  School. 

Provided  that  the  Pk  threshold  for  the  particular  orientation  of  the  launching  platform 
and  target  is  met,  the  weapon  will  fire.  Depending  on  the  sensor  characteristics  of  the  targeted 
platform  and  the  time  of  flight  of  the  weapon,  the  targeted  platform  may  choose  to  respond 
dynamically  in  a  fashion  that  suppresses  this  Pk.  For  example,  a  SAM  can  turn  its  radar  off 
upon  detection  of  a  missile  fire  and  thereby  dynamically  reduce  its  probability  of  being  killed 
by  antiradiation  missiles. 

This  model  can  assist  mission-level  analysis  in  any  vignette  in  which  weapons  are  employed 
by  an  airborne  platform,  such  as  close  air  support  (CAS)  of  ground  forces,  prosecution  of  time- 
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sensitive  targets,  and  in  SEAD  and  DEAD.  It  can  also  assist  in  any  vignette  that  is  placed  in  a 
contested  environment  with  air-to-ground  weapons. 
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Analytic  Methods  Were  Developed  to  Examine  the 
Factors  That  Affect  Operational  Effectiveness 


SCOPEM 


Platform 

•Orbit,  flight  path 
•Speed,  altitude 
•CONOPS 


Sensors 

•NIIRS  capabilities  (EO,  IR,  SAR, 
FMV) 

•RCS  resolution  (ISAR,  MTI) 
•Sensitivity,  scan  cycle  (SIGINT) 

Weapons 
•Range  and  speed 
•Accuracy  (CEP) 


Targets 

■  NIIRS  characteristics  (EO,  IR,  SAR) 

•  RCS  (ISAR,  MTI) 

•  ERP,  shape,  frequency  (SIGINT) 


Environment 
•Terrain  (LOS) 

•Sea  state,  wind 
•Atmosphere 
•Space  weather 


Survivability 

•  Permissive,  contested  air 
defenses 

•Air-to-air  threats 


Processing 

•Onboard  processing  (SWaP) 
•Communications,  PED  delays 


Environmental  factors  can  dramatically  change  the  operational  outcome  of  a  vignette  for  an 
otherwise  successful  candidate  platform  and  therefore  must  be  accounted  for  when  evaluating 
systems  at  the  mission  level  in  SCOPEM.  We  have  developed  a  suite  of  methods  to  address 
a  range  of  environmental  factors.  In  the  following  slides,  we  describe  our  methodologies  for 
evaluating  terrain  and  LOS,  sea  state  and  wind  in  the  maritime  domain,  atmospheric  condi¬ 
tions,  and  space  weather  effects  on  candidate  system  performance. 
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Terrain  and  Cloud  Cover  Models  Determine 
LOS  Based  on  Altitude 

•  DTED  and  SRTM  data  provide  terrain  altitudes 

-  Draw  ridgelines  to  eclipse  visibility  (PERL  script) 

-  Set  road  and  location  altitudes 

•  AFCCC  data  provide  cloud  coverage  and  altitude 

-  Set  cloud  ceiling  and  percentage  of  sky  obscured  (used  for  random 
draws  at  intervals) 


Ridgelines  Cloud  Cover 


Both  terrain  and  cloud  cover  can  block  the  LOS  between  a  sensing  RPA  and  the  target  of  inter¬ 
est  and  must  be  accounted  for  in  models.  Both  models  rely  on  large  data  sets  that  are  available 
for  Air  Force  research.  Digital  terrain  elevation  data  (DTED)  and  shuttle  radar  topography 
mission  (SRTM)  data  may  be  used  to  establish  terrain  altitude.  Air  Force  Combat  Climatol¬ 
ogy  Center  (AFCCC)  data  are  used  to  estimate  the  percentage  of  cloud  coverage  below  a  given 
altitude. 

LOSs  are  not  difficult  to  calculate,  but  the  computation  can  be  time-consuming.  More¬ 
over,  accurate  modeling  of  LOS  requires  similarly  accurate  models  of  terrain,  which  can  be 
very  challenging  to  generate,  especially  in  urban  environments.  For  this  reason,  we  represent 
terrain  in  two  ways.  In  general  terms,  we  use  a  practical  extraction  and  reporting  language 
(PERL)  script  to  process  DTED  at  the  relevant  longitude  and  latitude  to  create  a  set  of  ridge¬ 
lines  that  block  LOS.  These  are  also  used  to  determine  the  altitudes  of  any  locations  or  roads 
in  the  area.  The  altitudes  between  the  ridgelines  are  interpolated  in  a  straightforward  manner. 

For  urban  environments,  however,  we  adopt  a  different  approach.  Rather  than  modeling 
individual  buildings — which  SCOPEM  is  not  well-equipped  to  handle — we  instead  represent 
the  relevant  roads  as  “urban  canyons”  of  narrow  width  and  tall  walls.  This  finer  representation 
sits  on  top  of  the  ridgelines. 

Unlike  terrain,  cloud  cover  is  dynamic.  It  also  has  many  layers;  however,  for  our  purposes, 
we  need  only  consider  the  clouds  at  or  below  the  relevant  altitudes  of  the  aircraft.  We  are  not 
attempting  to  model  weather  but  to  represent  the  typical  effects  of  weather.  Using  AFCCC 
data,  we  divide  the  sky  into  a  grid.  The  percentage  of  the  sky  covered  by  clouds  is  given  for  that 
grid.  We  initially  populate  the  data  by  performing  a  random  draw  for  every  area.  After  a  given 
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time  (usually  on  the  order  of  an  hour  or  more),  we  redraw.  Critically,  we  perform  a  random 
draw  at  specified  intervals  rather  than  every  time  because  the  latter  method  would  permit  an 
aircraft  to  “see”  through  the  clouds  merely  by  repeated  attempts  at  sensing. 

Although  the  grid  spacing  and  redraw  intervals  are  properly  functions  of  the  spatial  and 
temporal  coherence  of  the  cloud  cover  in  that  area,  which  could  be  obtained  from  the  AFCCC 
data,  we  use  average  values  for  the  region  and  time  of  year  to  set  these  parameters.  Higher- 
resolution  modeling  of  subtle  variations  in  the  spacing  or  interval  is  not  required.  Because  that 
would  represent  a  higher  level  of  resolution  than  is  found  in  the  terrain  model,  it  would  get  lost 
in  the  noise.  Moreover,  because  these  distributions  are  not  correlated  with  anything  else  in  the 
model,  repeated  redrawing  over  multiple  Monte  Carlo  runs  would  be  expected  to  average  away 
these  higher-order  effects. 
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Space  Weather  Model  Represents  Daily  Variation  in 
Transmission  Loss  Through  Ionosphere 

•  Defines  several  shifting  regions  of  signal  degradation  for  several 
shifting  regions  of  ionospheric  activity 

-  Polar  areas  (auroral  clutter) 

-  Sun-following  regions  (scintillation  near  equator) 

-  SAA 


Used  to  give  dB  fade  depth  (reduction)  for  transmissions  and 
ELINT  detection  under  “normal”  and  “severe”  space  weather 
conditions 


/ 

Auroral  clutter 


Equatorial  scintillation 
SAA 


Space  weather  must  be  considered  when  examining  ISR  satellites  as  part  of  the  force  structure 
in  theater.  Space  weather  is  a  large  and  complicated  subject,  and  we  represent  only  a  small  slice 
of  it  within  SCOPEM.  Much  as  we  represented  only  one  feature  of  cloud  cover  (opacity),  we 
represent  one  feature  of  space  weather:  the  fading  depth  or  transmission  loss  (in  dB)  through 
the  ionosphere.  There  are  three  main  geographical  regions  of  concern:  the  auroral  clutter  at 
polar  latitudes  that  affects  radar  wavelengths,  a  sun-following  region  that  gives  rise  to  iono¬ 
spheric  scintillation  near  the  equator,  and  the  South  Atlantic  Anomaly  (SAA),  which  might 
affect  satellite  operation.5 

The  theory  behind  the  model  is  as  follows.  Roughly  speaking,  the  amplitude  of  the 
“noise”  in  the  ionosphere  due  to  solar  radiation  may  be  parameterized  by  the  S4  scintillation 
index.  The  Nakagami-M  distribution  links  this  to  the  fade  depth  (dB).  That  is,  for  a  given  S4, 
we  can  estimate  the  percentage  of  signals  we  expect  to  experience  at  a  fade  depth  of  at  least  a 
certain  level.6 

Using  information  provided  by  the  Air  Force  Research  Laboratory  (AFRL)  Space 
Weather  Center  of  Excellence,  we  estimate  approximate  values  of  S4  that  represent  “normal” 
and  “bad”  space  weather.  From  this,  we  use  a  representative  fade  depth  between  the  50th  and 
90th  percentiles,  meaning  that  most  of  the  worst  degradation  falls  into  this  category.  Typical 
values  near  the  equator  might  be  3  dB  to  10  dB.  These  fade  depths  are  then  used  to  create 


5  SAA  refers  to  the  area  where  the  earth’s  inner  Van  Allen  radiation  belt  is  closest  to  the  earth’s  surface. 

6  Nakagami-M  distribution  is  related  to  the  gamma  distribution.  S4  scintillation  index  characterizes  the  severity  of  ampli¬ 
tude  scintillation  based  on  radar  track  data. 
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moving  regions  of  signal  degradation.  At  present,  the  degradation  is  applied  uniformly  across 
the  region,  although,  as  the  discussion  above  indicates,  a  probabilistic  model  would  be  margin¬ 
ally  more  accurate. 

The  scintillation  index  is  applied  very  crudely  to  achieve  proper  scaling  for  the  effects.  We 
choose  this  simplistic  representation  because  it  is  sufficient  for  our  purposes:  If  space  weather 
is  raised  as  a  concern,  the  concern  is  likely  for  the  worst-case  scenario.  There  are  also  other 
important  effects  of  space  weather,  such  as  satellite  failures,  which  we  do  not  represent  using 
these  data.  Such  failures  would  be  modeled  instead  as  either  a  straightforward  blackout  or 
probability.  More-advanced  use  of  the  fade  depth  estimates  (such  as  to  compute  outages)  is  not 
warranted  by  this  methodology. 
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Maritime  Radar  Model  Represents  the  Effects 
of  Sea  and  Wind  on  the  Ability  to  Detect  Ships 


•  Skolnik’s  law  estimates  RCS  from  vessel  length 

•  Sea  clutter  model  incorporates  sea  state  and  wind  direction 

•  Minimum  RCS  thresholds  for  detection  and  classification  of  maritime 
vessels  at  range  by  ISAR  and  MMTI  are  based  on  sea  conditions 
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Skolnik’s  law  (RCS  versus  vessel 
length) 


Minimum  detectable  RCS  versus  slant  range 
(nmi)  (sea  states  1-6,  crosswind) 


Vessel  length  (ft) 


Slant  range  (nmi) 


The  maritime  radar  model  is  used  for  maritime  MTI  (MMTI)  detection  and  inverse  SAR 
(ISAR)  detection  and  classification.  We  describe  it  in  the  environmental  section  here  because 
the  wind  and  associated  sea  state  strongly  affect  the  radar  performance.  We  use  the  more  gener¬ 
alized  tracking  model  described  earlier  for  determining  suitable  surveillance  orbit  locations  to 
determine  whether  targets  will  remain  “in  track” — that  is,  whether  the  maritime  traffic  density 
is  such  that  confusion  between  ships  is  unlikely. 

In  this  model,  each  vessel  or  target  is  modeled  as  having  a  fixed  radar  cross-section  (RCS) 
based  on  its  length  (in  the  absence  of  more-detailed  data).  An  RCS  is  a  fictitious  area  that  rep¬ 
resents  the  strength  of  the  reflection  from  a  sphere  with  that  cross-section.  Thus,  the  RCS  may 
be  significantly  larger  or  smaller  than  the  actual  cross-sectional  area  presented  by  the  target.  In 
the  case  of  maritime  radar,  however,  the  echoes  from  a  vessel  vary  considerably  from  pulse  to 
pulse,  so  the  RCS  represents  a  further  level  of  averaging. 

We  make  several  simplifying  assumptions  regarding  target  RCS.  First,  we  use  the  same 
median  RCS  for  detection  regardless  of  the  orientation  of  the  vessel  with  respect  to  the  radar 
horn  or  the  height  of  the  vessel.  We  do  so  because  there  is  no  simple  relationship  between  RCS 
and  orientation  or  height  and,  generally  speaking,  information  of  greater  detail  that  would 
permit  determination  of  that  relationship  is  not  available.  Second,  despite  the  variance  of  the 
echo  mentioned  above,  we  use  the  median  value,  again  due  to  the  difficulty  of  estimating  the 
variance  with  sufficient  precision  to  merit  including  it.  Third,  we  ignore  the  mild  frequency 
dependence  of  the  RCS.  Although  the  order  of  magnitude  remains  the  same,  the  equivalent 
RCS  for  X-band  radar  should  be  somewhat  smaller  than  that  for  S-band. 
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The  crucial  part  of  the  model  is  the  calculation  of  the  minimum  detectable  RCS  based 
on  sea  state  and  wind  direction.  Sea  state  is  measured  by  the  international  Douglas  Sea  Scale 
(World  Meteorological  Organization,  2006).  It  is  an  integer  from  0  to  9  that  represents  the 
character  and  height  of  the  waves.  Typical  sea  states  for  each  season  are  publicly  available  for 
most  locations;  we  use  National  Oceanic  and  Atmospheric  Administration  data  to  set  approxi¬ 
mate  sea  regions.  The  average  wind  direction  and  strength  are  also  available.  A  complex  sea 
clutter  model  provides  different  minimum  RCS  versus  slant  range  curves  for  each  sea  state,  for 
both  downwind  and  crosswind  conditions  (direction  relative  to  the  sensor).  These  curves  must 
be  generated  separately  for  each  radar. 
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MODTRAN  Is  Employed  to  Model  the  Effects  of 
Weather  on  EO/IR  Sensor  Capability 

•  MODTRAN  is  a  Fortran-based  program  developed  by  AFRL 

-  Used  to  determine  the  attenuation  (transmittance)  of  EO/IR  radiation  through  the 
atmosphere  for  varying  weather  conditions  and  climates 

-  May  include  various  particulates  and  precipitation:  e.g.,  volcanic  ash,  snow,  fog 

-  Contains  “typical”  weather  conditions  for  specific  regions  at  specific  times,  e.g., 

“Persian  Gulf  in  August” 

*  Atmospheric  transmittance  is  used 
to  determine  NIIRS  degradation  due 
to  diverse  weather  and  atmospheric 
conditions 

-  SCOPEM  employs  MODTRAN- 
generated  look-up  tables  to  determine 
the  transmittance  through  the 
atmosphere  for  a  given  look  angle, 
range  to  target,  and  sensor  type 

-  The  SNR  term  in  the  GIQE  is  modified 
based  on  this  transmittance: 

SNR^R^ )  =  SNR,m^,Rj~\ 


Atmospheric  Transmittance  Through  Fog 
(visibility  1,220  meters) 


(Mid-latitude,  summer  and  rural  aerosols) 


MODTRAN  is  a  Fortran-based  program  developed  by  the  AFRL  to  determine  the  trans¬ 
mittance  of  visible  and  IR  radiation  through  different  atmospheres  and  weather  conditions. 
MODTRAN  has  the  capability  to  calculate  the  effect  of  particulates  and  precipitation,  includ¬ 
ing  snow,  fog,  rain,  and  volcanic  ash,  on  radiation  transmittance.  An  auxiliary  graphical  user 
interface,  PcModWin,  provides  access  to  libraries  of  “typical”  weather  conditions  for  specific 
regions,  at  specific  times,  around  the  world.  MODTRAN  calculates  the  transmittance  of  radi¬ 
ation  at  visible  and  IR  wavelengths  at  different  slant  ranges  and  look  angles  for  each  weather 
and  atmospheric  condition. 

PAF  uses  MODTRAN  (via  PcWinMod,  versions  4  and  5)  to  simulate  the  effects  of 
weather  on  sensor  performance.  Specifically,  atmospheric  transmittance,  or  attenuation,  from 
MODTRAN  is  used  to  determine  the  associated  degradation  in  attainable  sensor  NIIRS  level 
in  SCOPEM.  The  average  SNR  term  (as  described  previously)  in  the  GIQE  will  be  scaled  based 
on  reduced  transmittance  of  IR  and  visible  wavelengths  through  the  atmosphere.  For  example, 
the  scaling  factor  for  a  foggy  day  is  determined  by  dividing  the  transmittance  through  fog  by 
the  transmittance  on  a  clear  day  for  a  given  look  angle,  slant  path,  and  wavelength  such  that 

SNRf,  (R„., )  =  SNR 

standard 

where  tfig  is  transmittance  through  a  defined  radiative  fog  atmosphere  and  t  is  an  average 
transmittance  through  a  standard  atmosphere  on  a  clear  day. 
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RAND’S  SCOPEM 


SCOPEM 


Platform 

•Orbit,  flight  path 
•Speed,  altitude 
•CONOPS 


f 

Survivability  models 
•Air  intercept  model 
•TAC  Brawler 
•ESAMS 
•MOSAIC 
•RADGUNS 

•Aural  detection  model 
•Visual  detection  model 
•Suppressor 

\ 

Environment 
•Terrain  (LOS) 

•  Sea  state  /  Wind 

•  Atmosphere 

•  Space  weather 


Survivability 
•Air-to-air  threats 

•  Surface-to-air  threats 

•  Integrated  air  defenses 


Targets 

NIIRS  characteristics  (EO,  IR,  SAR) 
RCS  (ISAR,  MTI) 

ERP,  shape,  frequency  (SIGINT) 


Processing 

•  Onboard  processing  (SWaP) 

•  Communications,  PED  delays 


Candidate  platforms  operating  in  contested  or  denied  threat  environments  may  face  advanced 
air  defenses,  including  double-digit  SAMs  and  air-to-air  threats.  We  now  discuss  the  surviv¬ 
ability  analysis  needed  for  these  challenging  circumstances  and  how  our  survivability  analysis 
provides  input  into  our  mission-level  model  (SCOPEM). 
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Survivability  Analysis  Relies  on  Aggregated  Results 
Generated  by  Detailed  Engagement  Models 

*  RPA  survivability  is  a  function  of 

-  Platform  performance:  Airspeed,  altitude,  maximum  turn  rate 

-  Signature:  Radar,  IR,  visual,  aural,  passive  RF 

-  Countermeasure  payload:  Jammers,  threat  warning  system,  chaff/flare,  towed  decoys 

*  Detailed  models  capture  key  elements  of  threat  engagements 

*  SCOPEM  uses  aggregated  results  from  detailed  models  to  reflect  survivability 


System  Inputs 

RPA  Description 

•  Aero  performance 

•  Signatures 

•  Countermeasure 
payload 

Threat  Description 

•  Type  of  sensor 

•  Type  of  threat 


Engagement  Models 


Cuing  Models 

•Visual 

•Aural 

•Passive  RF 


Air-to-air  model 
•  Brawler 


Air  defense  models 

•  AAA  (RADGUNS) 

•  IR  SAM  (MOSAIC) 

•  RF  SAM  (ESAMS) 
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Downrange 

SCOPEM 

Inputs 


•  Detection  curve 

*  Availability  factors 


Pk  envelope 
Missile  flyout  time 
Engagement  delay 
Weapon  load 
Availability  factors 


Aircraft  survivability  is  a  function  of  many  factors,  including  characteristics  of  the  aircraft  and 
weapon  system,  effectiveness  of  countermeasures,  and  the  CONOPS  for  achieving  mission 
objectives.  Although  some  models  excel  at  simulating  one-on-one  engagements  with  sensors 
or  air  defenses,  other  models  excel  at  capturing  overall  mission  and  campaign  effectiveness. 
Models  focused  on  one-on-one  engagements  typically  offer  high  resolution  and  high  detail 
but  tend  to  be  narrow  in  scope.  By  contrast,  mission-level  and  campaign-level  models  are 
broader  in  scope  and  attempt  to  capture  multiple  aircraft  and  weapon  systems  in  a  complex 
environment. 

PAF’s  approach  to  survivability  analysis  involves  running  multiple  high-resolution  models 
that  capture  the  specific  factors  that  determine  survivability.  The  results  of  these  high-resolution 
models  are  used  for  specific  point  design  analyses  and  translated  into  formats  that  can  be  read 
by  mission-level  models,  such  as  SCOPEM,  which  are  less  detailed  but  comprehensive  enough 
to  perform  mission-level  analysis. 

To  properly  analyze  survivability,  PAF’s  methodology  addresses  platform  performance, 
signatures,  and  potential  countermeasures.  Platform  performance  can  be  described  by  the 
altitude,  airspeed,  and  maneuverability  (i.e.,  maximum  turn  rate)  of  the  aircraft.  Signatures 
include  RCS,  IR  signature,  presented  area  for  visible  detection,  and  sound  characteristics  for 
aural  detection.  In  the  case  of  RPAs,  the  characterization  of  the  communication  link  must  also 
be  evaluated  because  passive  radio  frequency  (RF)  sensors  may  detect  the  data  communica¬ 
tions  required  for  controlling  the  platform  and  transmitting  data.  Depending  on  the  space  and 
power  available,  an  aircraft’s  survivability  may  be  improved  by  employing  a  threat  warning 
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receiver,  jammers,  flare/chaff  dispensers,  or  towed  decoys.  PAF  has  experience  with  a  suite  of 
models  and  tools  designed  to  address  all  of  these  critical  survivability  characteristics. 

The  following  models  are  currently  available  for  conducting  RPA  survivability  analysis. 
We  discuss  each  in  the  following  slides:7 

•  detection  engagement  modeling 

-  aural  detection  model 

-  visual  detection  model 

•  air-to-air  model 

-  Tactical  Air  Campaign  (TAC)  Brawler 

•  air  defense  engagement 

-  ESAMS 

-  Modeling  System  for  the  Advanced  Investigation  of  Countermeasures  (MOSAIC) 

-  Radar-Directed  Gun  Simulation  (RADGUNS). 


Background  on  each  model  is  provided  in  “Additional  Detail  on  Selected  Models”  at  the  end  of  this  briefing. 
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Air  Intercept  Phases  of  Conflict 


Threat  CAP 
station 


Autonomous 
detection  by 
CAP  station 


Reassign  ? 
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Phase 

Phase  Title 

1 

No  GCI  detection 

2 

GCI  detection 

3 

GCI  track 

4 

C3  delay 

5 

Airbase  delay 

6 

Climb 

7 

Cruise 

8 

Al  detection 

9 

BVR  combat 

10 

WVR  combat 

11 

Reassign 

12 

Return  to  base 

13 

Maintenance 

14 

Refuel  and  rearm 

One  of  the  biggest  threats  to  the  survival  of  future  RPAs  is  the  air-to-air  threat.  Threat  air-to- 
air  fighters  can  either  be  on  the  ground  at  an  airbase  or  in  the  air  on  combat  air  patrol  (CAP). 
Shown  in  this  slide  is  our  representation  of  this  threat.  The  table  on  the  right  lists  the  phases 
of  an  air-to-air  intercept. 

In  phase  1,  the  RPA  has  not  been  detected  by  a  ground  control  intercept  (GCI)  radar 
or  a  threat  Airborne  Warning  and  Control  System.  As  soon  as  a  GCI  detects  the  RPA,  the 
intercept  goes  into  phase  2  (GCI  detection).  The  GCI  must  detect  the  RPA  two  times  out  of 
three  scans  before  the  GCI  operator  declares  a  GCI  track  (phase  3).  Once  a  GCI  track  has 
been  established,  a  command,  control,  and  communications  (C3)  delay  begins  (phase  4).  This 
delay  represents  the  time  it  takes  from  the  establishment  of  a  GCI  track  to  the  time  the  sector 
operations  center  orders  the  aircraft  to  scramble  from  an  airbase.  Once  the  scramble  order  is 
received,  there  is  a  taxi  and  takeoff  delay  (phase  5).  The  interceptor  aircraft  then  climbs  to  the 
cruise  altitude  (phase  6)  and  cruises  out  toward  the  RPA  (phase  7).  The  airborne  radar  must 
detect  the  RPA  two  scans  out  of  three  to  establish  a  detection  (phase  8).  If  an  interceptor  is 
on  CAP,  it  can  autonomously  detect  the  RPA  and  would  begin  the  intercept  mission  at  this 
point  (phase  8).  Once  the  interceptor  has  established  a  track,  it  can  employ  beyond-visual- 
range  (BVR)  missiles  and  within-visual-range  (WVR)  missiles  (phases  9  and  10,  respectively). 
If  the  RPA  is  destroyed,  the  interceptor  can  either  be  vectored  toward  another  RPA  (phase  11) 
or  return  to  base  (phase  12),  where  it  can  refuel  and  rearm  (phase  14)  and  possibly  conduct 
another  sortie. 

The  above  model  is  a  stand-alone  spreadsheet  model.  Many  of  the  inputs  are  taken  from 
TAC  Brawler.  The  Air  Force  model,  Brawler,  is  a  high-resolution  model  that  represents  air- 
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borne  threats  to  blue  airplanes.  Metrics,  such  as  red/blue  kill  ratios,  are  Brawler  outputs  that 
feed  into  SCOPEM  as  look-up  tables. 
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Visual  and  Aural  Detection  May  Provide  Initial 

Cues  to  Air  Defenses 


Cumulative  Cross-range 


Maximum 

detectable 

range 


The  visual  and  aural  detection  models  represent  detection  by  human  observers.  Although  not 
the  primary  early  detection  mechanisms  for  integrated  air  defenses  with  early  warning  (EW) 
radars,  human  observers  were  used  during  Operation  Allied  Force  in  Serbia.  Observers  may  be 
cued  to  the  general  direction  of  approaching  aircraft,  or  they  may  be  scanning  the  horizon  for 
targets.  For  visual  detection,  the  observer  can  be  aided  by  optics  (e.g.,  binoculars)  or  may  be 
reliant  on  the  naked  eye.  Aural  detection  may  also  be  treated  as  aided  or  unaided,  but  aircraft 
engine  noise  must  rise  above  competing  ambient  noise  levels  to  have  a  chance  of  being  heard. 

The  detailed  aural  and  visual  detection  models  can  be  used  to  evaluate  the  effect  of  specific 
RPA  platform  design  choices  under  varied  environmental  conditions.  Airframe  size,  shape,  and 
color  have  a  direct  impact  on  visual  detection  range  in  our  detailed  detection  model.  Similarly, 
engine  choice  and  placement  on  an  airframe  have  a  direct  impact  on  detection  range  in  our 
aural  model.  Environmental  conditions,  such  as  haze,  cloud  cover,  foliage,  and  terrain  features, 
are  also  captured  in  the  visual  and  aural  detection  models.  Outputs  from  these  high-resolution 
models  can  then  be  used  as  inputs  into  SCOPEM,  allowing  specific  visual  and  aural  detection 
capabilities  to  manifest  effects  on  mission-level  analysis  and  trade-offs  across  CONOPS. 

The  visual  and  aural  modeling  results  are  translated  into  SCOPEM  inputs  through  a 
series  of  tables  containing  detection  probabilities.  By  aggregating  the  results  of  many  runs  of 
the  visual  and  aural  detection  models,  analysts  can  produce  curves  describing  cumulative  PD 
for  various  combinations  of  downrange,  cross-range,  and  altitude.  SCOPEM  can  refer  to  these 
detection  tables  for  PD  and  estimate  detection  ranges.  For  points  falling  between  (or  beyond) 
the  given  set  of  probability  curves,  SCOPEM  would  use  interpolation  and  extrapolation  meth¬ 
ods  to  estimate  probabilities. 
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Modeling  a  SAM  Engagement  Is  a  Four-Step 

Process 


1)  Calculate  entry  into 
engagement  envelope. 


2)  Calculate  intercept 
time  based  on  flyout 
curve. 


3)  Assess  kill  based  on 
random  draw  and  Pk. 


Downrange 


4)  If  aircraft  survives,  incur 
reengagement  delay  and  test 
for  new  opportunity  to  engage. 


SAM 

envelope 


Cells  in  SAM  envelope 
reflect  Pk  of  engagement 


Inputs  to  SCOPEM: 
Horizontal  SAM 
envelopes  with  Pk 
Missile  flyout  curve 
Reengagement  delay 
Weapon  load 
Threat  availability  factor 


PAF  has  significant  experience  modeling  air  defenses,  including  EW  radars,  antiaircraft  artil¬ 
lery  (AAA),  IR  SAMs,  and  RF  SAMs.  AAA  systems  are  modeled  using  the  RADGUNS 
model,  IR  SAMs  are  modeled  using  MOSAIC,  and  RF  SAMs  are  modeled  using  ESAMS. 
High-end  integrated  air  defenses  with  EW  and  RF  SAM  radars  linked  together  are  modeled 
using  Suppressor.  Each  of  these  models  is  currently  in  use  by  the  Air  Force.  Though  the  format 
of  results  from  each  of  these  models  varies,  the  results  of  each  model  can  be  translated  into  a 
common  format  that  can  be  interpreted  by  SCOPEM  to  represent  air  defenses.  By  computing 
one-on-one  engagement  outcomes  for  various  combinations  of  downrange,  cross-range,  and 
altitude,  each  model  can  produce  data  sets  to  build  horizontal  Pk  envelopes.  Each  Pk  envelope 
is  made  up  of  multiple  cells,  with  each  cell  depicting  the  Pk  of  an  aircraft  at  a  specific  down- 
range,  cross-range,  and  altitude  combination.  To  implement  these  envelopes  in  SCOPEM, 
additional  data  are  required  to  supplement  the  Pk  envelopes.  SCOPEM  also  requires  missile 
flyout  curves  that  describe  the  distance  flown  for  each  increment  in  time,  a  reengagement  delay 
to  reflect  the  time  between  shots,  a  weapon  load  to  establish  the  weapon  system’s  capacity  to 
fire  on  aircraft,  and  an  availability  factor  to  reflect  the  probability  that  the  threat  is  in  use  for 
any  particular  trial  executed  in  SCOPEM. 

To  use  the  Pk  envelopes,  SCOPEM  first  calculates  the  point  at  which  the  aircraft  enters 
the  Pk  envelope.  At  that  point,  SCOPEM  uses  the  missile  flyout  curve  to  determine  an  inter¬ 
cept  time  and  location.  Using  a  random  draw  from  0  to  1,  SCOPEM  compares  the  random 
draw  with  the  Pk  reflected  in  the  engagement  cell  to  determine  whether  the  aircraft  has  been 
shot  down.  If  the  random  draw  is  less  than  the  Pk,  the  aircraft  is  presumed  killed.  If  the 
random  draw  is  higher  than  the  Pk,  SCOPEM  assesses  an  engagement  delay,  then  uses  the 
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aircraft’s  updated  position  to  determine  whether  another  engagement  is  possible.  If  another 
engagement  is  possible,  SCOPEM  repeats  the  engagement  calculations  based  on  the  updated 
Pk  envelope  and  threat  parameters.  SCOPEM  continues  this  process  until  the  aircraft  is  killed, 
the  aircraft  leaves  the  Pk  envelope,  or  the  weapon  system  runs  out  of  weapons. 
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Analytic  Methods  Were  Developed  to  Examine  the 
Factors  That  Affect  Operational  Effectiveness 


SCOPEM 


Platform 

•Orbit,  flight  path 
•Speed,  altitude 
•CONOPS 


Sensors 

•NIIRS  capabilities  (EO,  IR,  SAR, 
FMV) 

•RCS  resolution  (ISAR,  MTI) 
•Sensitivity,  scan  cycle  (SIGINT) 

Weapons 
•Range  and  speed 
•Accuracy  (CEP) 


Targets 

■  NIIRS  characteristics  (EO,  IR,  SAR) 

•  RCS  (ISAR,  MTI) 

•  ERP,  shape,  frequency  (SIGINT) 


Environment 
•Terrain  (LOS) 

•Sea  state,  wind 
•Atmosphere 
•Space  weather 


Survivability 

•  Permissive,  contested  air 
defenses 

•Air-to-air  threats 


Processing 

•Onboard  processing  (SWaP) 
•Communications,  PEP  delays 


Conveying  data  or  intelligence  between  the  candidate  platform  and  other  platforms  or  com¬ 
mand  centers  affects  the  operational  effectiveness  of  the  candidate  platform.  We  have  devel¬ 
oped  two  tools  and  leverage  a  third  RAND  tool  to  model  these  factors.  The  first  evaluates 
the  onboard  processing  requirements  for  a  mission;  the  second  models  the  communications 
network.  The  third  characterizes  processing,  exploitation,  and  dissemination  (PED)  delays  for 
surveillance  and  reconnaissance  data.  The  first  model  requires  input  from  SCOPEM.  From 
the  mission-level  outcome  of  SCOPEM,  the  associated  data-collection  requirement  can  be  fed 
into  our  onboard  processing  tool  to  evaluate  the  onboard  processing  required  to  implement 
this  candidate  in  the  mission  evaluated.  The  second  model  also  uses  data  from  SCOPEM  to 
evaluate  the  communications  network  necessary  to  employ  RPAs  in  a  given  mission.  The  third 
model  provides  SCOPEM  with  a  means  of  capturing  the  delays  associated  with  sending  and 
processing  data.  The  next  three  slides  describe  these  tools. 
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SWaP  and  Onboard  Processing  Requirements  Constrain  Total  Sensor  Payloads 

FLOPS/watts  projection  through  2025 
Date 

10/1990  04/1996  09/2001  03/2007  09/2012  03/2018  06/2023 

100000000 


0  1 


Analysis  of  FLOPS  for  target  tracking  algorithms  2020 


Sensor  Type 

Kalman  Filter 
GFLOPS 

Kalman  Filter 
Power 

Bootstrap  Particle 
GFLOPS 

Bootstrap  Particle 
Power 

Grayscale  FMV  5  Hz 

50 

17  W 

50 

17  W 

Multicolor  FMV  5  Hz 

450 

150  W 

150 

50  W 

Gorgon  Stare  12-channel  2  Hz 

600 

200  W 

240 

80  W 

Gorgon  Stare  60-channel  2  Hz 

3,000 

1  kW 

1,200 

400  W 

200-band  hyperspectral  1  Hz 

400,000 

133  kW 

2,000 

670  W 

Because  of  bandwidth  and  personnel  head-count  constraints  to  support  PED  activities, 
onboard  computing  is  likely  to  become  increasingly  important  to  the  effectiveness  of  sensor 
exploitation  on  future  candidate  RPAs.  Computing  requirements  may  scale  nonlinearly  with 
respect  to  sensor  resolution  and  data  throughput.  Not  only  has  computer  chip  performance 
increased  exponentially  historically,  the  electrical  power  consumption  of  microprocessors  has 
risen  as  well. 

In  this  slide,  we  have  compiled  data  on  34  desktop  central  processing  units  (CPUs)  from 
1990  to  the  present  and  evaluated  how  the  number  of  floating-point  operations  per  second 
(FLOPS)  and  power  consumption  per  chip  have  increased  over  time.  Although  the  number 
of  FLOPS  per  chip  doubles  every  18  months  in  accordance  with  Moore’s  law,8  power  con¬ 
sumption  increases  more  slowly,  so  that  the  number  of  FLOPS  per  watt  (W)  doubles  every 
33  months.  Because  the  amount  of  electrical  power  on  an  RPA  is  likely  to  be  closely  related  to 
its  total  size,  available  power  will  eventually  become  the  limiting  factor  in  onboard  comput¬ 
ing,  at  which  point  advances  in  processing  power  on  RPAs  will  likely  be  slower  than  expected 
under  Moore’s  law.  Plotted  on  the  graph  are  contours  of  FLOPS  versus  time  if  processing  is 
limited  by  power  levels  of  10  W,  100  W,  and  2,000  W. 

To  demonstrate  what  increased  onboard  computing  power  can  do  in  the  future,  we  look 
at  a  challenging  sensor  application  that  will  require  or  make  use  of  large  amounts  of  onboard 
computing  power:  autonomously  tracking  a  target  using  FMV.  We  then  evaluate  the  computa- 


Moore’s  law  states  that  the  number  of  transistors  on  a  chip  will  double  approximately  every  two  years. 
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tional  requirements  for  popular  algorithms  used  to  address  this  class  of  problems  and  scale  the 
requirements  to  operationally  relevant  situations.  These  requirements  can  then  be  converted 
from  FLOPS  to  watts  using  historical  trend  lines  from  desktop  CPUs  for  a  variety  of  current 
and  future  FMV  sensors. 

As  predictions  of  the  future,  these  estimates  are  necessarily  speculative,  but  they  can  give 
order-of-magnitude  projections  of  which  sensor  capabilities  are  feasible  for  autonomous  opera¬ 
tion  and  can  have  important  implications  for  sensor  payloads  and  processing  and  communica¬ 
tion  requirements. 
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RPA  Communications  Network  Performance  Can  Be 


Estimated  Using  RAND’s  NEAT 


□  Microsoft  Access 

BBS 

File  Edit  View  Insert  Format  Records  Tools  Window  Help 

41  if  VI  M  B4’'£ 

Type  a  qu 

a. 

Z1 

Access  database 
for  storing  data 
about  large-scale 
networks 


Front-end  and  high- 
performance  number 
crunching  developed  in  C++ 


Novel  approach  based  on 
math  programming  to 
estimate  performance 
rather  than  modeling 
physics  of  actual  network 
Can  quickly  estimate 
performance  of  very 
large  networks 
Developed  with  internal 
funds,  runs  on  RAND’s 
high-performance 
computing  cluster 


SOURCE:  Gonzales  et  al.,  2010. 


The  network  exploratory  analysis  tool  (NEAT),  which  is  not  part  of  SCOPEM,  is  a  tool 
designed  to  estimate  the  impacts  of  architectural  changes  to  network  MOPs.  MOPs  include 
statistics  on  throughput  rates,  packet  loss  ratios,  statistics  on  message  delays,  and  the  number 
of  hops  needed  to  deliver  messages.  It  also  enables  an  analyst  to  identify  key  bottlenecks  within 
an  architecture  where  the  MOPs  are  limited.  Conventional  network  modeling  tools  do  this  by 
implementing  routing  algorithms  and  simulating  the  physics  of  a  network,  but  a  drawback  of 
this  conventional  approach  is  that  simulation  run  time  increases  dramatically  with  the  size  of 
the  network  as  measured  by  the  number  of  nodes  and  arcs  used  to  represent  it.  NEAT  uses 
a  novel  approach  that  avoids  representing  the  entire  network  during  event-stepped  simula¬ 
tion  and  thereby  reduces  the  computational  loading  and  facilitates  rapid  simulation  of  very 
large  networks.  The  approach  is  based  on  a  mathematical  optimization-based  routing  scheme 
that  requires  representing  only  a  single  route  during  discrete  event  steps.  For  more  details,  see 
Gonzales  et  al.,  2010. 
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Queuing  Model  Provides  Estimates  of  Time 
Required  to  Complete  PED  Processes 

•  Flexible  C++  model  of  queuing  processes  for  IMINT 

•  Uses  queuing  theory  to  estimate  image  exploitation  time 
distribution  based  on  input  loading  and  workflow  structure 


Queues  Analysts 


The  PED  process  is  not  represented  explicitly  in  SCOPEM  but  in  a  separate  queuing  model. 
We  are  interested  in  the  effects  of  the  PED  process  on  the  rest  of  the  operations.9  The  primary 
impact  is  simply  that  the  PED  process  requires  valuable  time,  particularly  for  imagery  intel¬ 
ligence  (IMINT).10  The  queuing  model,  programmed  in  C++,  accepts  a  certain  rate  of  input 
(incoming  images)  and  returns  a  distribution  of  times  from  which  we  can  draw  to  determine 
how  long  the  PED  process  takes  for  each  image.  Alternatively,  we  can  use  the  average  or 
median  time  of  this  distribution. 

The  queuing  model  allows  us  to  examine  the  benefits  of  different  PED  organizational 
structures.  In  this  slide,  each  analyst  is  assigned  a  set  of  images  and  works  through  his  or 
her  pile  of  images  separately.  A  second  option  would  be  to  use  a  single  queue  and  allow  each 
analyst  to  draw  the  next  available  image  from  the  list.  Targets  detected  by  EO/IR  sensors  are 
held  onboard  the  aircraft,  or  at  the  ground  station,  until  the  PED  processing  time  determined 
above  has  elapsed.* 11  Then  they  are  released  for  cuing  and  further  action.  This  allows  us  to  rep¬ 
resent  in  the  larger  model  the  limits  this  places  on  Air  Force  operations.  For  example,  a  time- 
sensitive  target  that  must  be  identified  may  no  longer  be  visible  by  the  time  the  identification 
is  complete. 


9  According  to  recent  articles,  the  manpower  for  exploitation  of  a  single  Predator  feed  is  19  analysts  (e.g.,  “US  Air  Force 
Funds  $86M  Blue  Devil  2  Demonstration  Airship,”  2011). 

10  SCOPEM  can  similarly  account  for  additional  command  and  control  (C2)  or  PED  delays  by  implementing  a  delay  in 
the  operational  model.  Of  course,  the  model  is  very  dependent  on  the  quality  of  representative  data  for  these  delays. 

11  There  is  no  queuing  model  for  other  intelligence  data,  such  as  SIGINT;  for  those,  we  use  a  single  typical  time. 
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By  representing  the  PED  process,  we  can  examine  the  effectiveness  of  an  ISR  fleet  as  a 
function  of  the  size  and  organizational  structure  of  the  PED  enterprise. 
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RPA  Evaluation  Methodology 


Once  analysis  for  a  candidate  system  in  a  vignette  under  varying  conditions  is  complete,  the 
effectiveness  matrix  can  be  populated.  We  next  discuss  the  effectiveness  matrix  by  looking  at 
an  example. 
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Effectiveness  Matrix  Describes  Conditions 
Under  Which  the  System  Succeeds 

•  Range  of  environmental  factors  affect  success 

*  Should  also  note  the  number  of  such  systems  required 


Notional  Effectiveness  Matrix 


Vignette 

Force 
structure 
option  1 

Force 
structure 
option  2 

Force 
structure 
option  3 

1.  Overwatch  and  CAS  to  defend  an 
outpost 

<30  enemies 
Daylight 

Daylight 
<45  enemies 

2+  platforms 

No  success 

2.  Detect  and  track  an  HVT  across 
mountains  and  urban  terrain 

Low  traffic 
<25  km  avg. 

3+  platforms 

No  confusers 

Any  nighttime 
engagement 

3.  Find  and  neutralize  time-sensitive 
targets  (SEAD/DEAD) 

No  clouds 
Daylight  only 

<3  SAMs 

Permissive  or 

denied 

environment 

All  weather 

<30  SAMs 

The  end  result  of  the  mission-level  analysis  should  be  an  effectiveness  matrix  that  lays  out  the 
various  missions  (or  vignettes)  and  describes  how  well  each  candidate  system  performed  in  that 
situation  or  under  what  conditions  the  system  was  able  to  succeed,  including  the  number  of 
platforms  needed  and  any  environmental  restrictions  (e.g.,  daytime  only). 

The  effectiveness  matrix  maps  out  the  trade  space  to  illuminate  the  strengths  and  limi¬ 
tations  of  different  system  choices.  Consider  the  notional  effectiveness  matrix  shown  for  the 
first  vignette  listed  in  the  slide.  Systems  1  and  2  succeed  in  daylight  conditions  for  providing 
overwatch  and  CAS  to  defend  an  outpost.  System  3  fails.  However,  SCOPEM  analysis  shows 
that  system  2  requires  two  platforms  to  perform  the  task,  while  system  1  requires  only  one 
platform.  Still,  system  2  was  more  successful  at  locating  and  killing  enemy  combatants  (i.e.,  45 
versus  30  enemy  combatants).  Thus,  the  analyst  and  the  decisionmaker  see  the  range  of  trade¬ 
offs  involved  in  using  either  system.  Ultimately,  this  kind  of  analysis  would  feed  into  a  cost- 
effectiveness  analysis  that  would  compare  the  relative  cost  of  one  system  1  platform  with  that 
of  the  two  system  2  platforms  needed  to  succeed  in  this  mission.  To  understand  the  number 
of  candidate  platforms  necessary  over  a  larger  campaign,  a  campaign-level  model,  such  as 
RAND’s  FSE,  as  discussed  in  subsequent  slides,  is  necessary  to  establish  an  overall  force  struc¬ 
ture  requirement. 
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Example  Vignette:  Track  an  HVT  in  Mountainous  and 

Urban  Terrain 

•  Mission  is  to  track  HVT  from  airport  to  destination 

-  HVT  leaves  airport  at  some  point  during  24-hour  interval 

-  HVT  may  choose  one  of  three  destinations  randomly 


To  illustrate  the  method  in  more  detail,  we  ran  a  test  case  from  beginning  to  end  to  compare 
two  notional  alternative  RPAs  in  one  vignette.  The  mission  was  to  track  an  HVT  driving 
from  an  airport  to  its  final  destination.  The  time  of  arrival  is  not  precisely  known;  the  vignette 
assumes  that  the  intelligence  specifies  only  that  the  HVT  is  to  arrive  at  some  point  during  a 
24-hour  interval.  Likewise,  the  destination  is  unknown.  The  ultimate  mission  is  to  identify  the 
destination  by  following  the  HVT. 

We  considered  three  different  cases  of  the  vignette,  corresponding  to  three  different  pos¬ 
sible  destinations.  Each  case  was  run  1,000  times  to  reflect  various  combinations  of  environ¬ 
mental  factors.  We  considered  clouds,  fog,  and  clear  skies;  because  the  airport  arrival  time  is 
not  known,  daylight  and  nighttime  conditions  were  simulated  with  roughly  equal  probability 
as  well.  The  terrain  was  mountainous,  and  part  of  the  route  went  through  an  urban  area,  so 
both  aspects  of  the  terrain  model  were  used. 

The  simulation  begins  with  the  RPA  circling  the  airport  at  a  discreet  distance,  using 
video  surveillance  to  watch  for  the  HVT  to  emerge  in  a  vehicle.  We  assumed  that  the  vehicle 
was  known.  The  HVT  was  assumed  to  travel  with  the  speed  of  traffic  (which  varied  along  the 
route)  to  its  destination.  The  CONOPS  was  that  the  RPA  had  to  positively  identify  the  HVT 
vehicle  before  following  it.  If  the  track  was  lost,  the  RPA  had  to  repeat  the  identification  pro¬ 
cess  successfully  before  it  could  reacquire  the  track.  The  RPA  was  allowed  to  scan  ahead  on  the 
highway  (in  the  rural  areas)  if  the  track  was  lost.  If  the  target  was  never  acquired,  or  the  track 
was  lost,  the  RPA  was  deemed  to  have  failed  the  mission.  Only  if  the  HVT  was  tracked  all  the 
way  to  its  final  destination  was  it  considered  a  success.  The  MOE  was  therefore  binary. 
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Example  of  Effectiveness  Matrix  for  the 
“ Tracking  an  HVT”  Vignette 


Candidate  platform  characteristics  (notional) 


Candidate  A 

Candidate  B 

Parameters 

25,000  ft,  120  mph 

30,000  ft,  300  mph 

Sensors 

MTS-B,  MP-RTIP 

MTS-B,  Lynx 

Weapons 

Hellfire 

Hellfire,  GBU 

Percentage  of  time  the  platform  was  able  to  follow  HVT  all  the  way  from 
the  airport  to  the  destination  (if  track  is  lost  or  not  acquired,  mission 


fails) 

Candidate  A 

Candidate  B 

Clear 

98% 

75% 

Cloudy 

35% 

2% 

Fog 

0% 

0% 

Requirement  to  identify  HVT  prior  to  pursuit  implies 
visual  image  required,  hence  failure  due  to  weather/fog 


The  candidate  platform  characteristics  and  results  of  the  simulation  are  shown  in  this  slide. 
(The  speeds  and  altitudes  shown  here  are  approximate.)  Both  platforms  had  the  same  FMV 
capability  but  used  different  GMTI  sensors.12  The  weaponry  was  different  but  was  not  used  in 
this  particular  scenario.  In  this  vignette,  the  comparative  effectiveness  of  the  two  platforms  was 
driven  by  two  factors.  First,  the  radar  for  candidate  B  was  not  as  powerful,  so  track  was  lost 
more  often,  even  under  clear  conditions.  Second,  the  requirement  to  identify  the  HVT  before 
tracking  meant  that,  if  LOS  could  not  be  established,  as  in  fog  or  under  clouds,  the  mission 
could  not  succeed. 

These  results  inform  the  analyst  that,  under  cloudy  conditions,  neither  candidate  is  par¬ 
ticularly  effective,  while  candidate  A  is  preferred  over  candidate  B  for  following  H  VTs  in  clear 
weather.  Adding  cost  analysis  to  these  results  informs  decisionmakers  on  the  cost  and  effec¬ 
tiveness  of  the  two  candidates.  Inputting  these  results  into  a  campaign-level  model,  such  as 
RAND’s  FSE,  will  further  inform  decisionmakers  as  to  the  appropriate  candidate  force  size 
needed. 


12  The  multispectral  targeting  system  for  Predator  B  (MTS-B)  is  the  FMV  sensor  on  board  the  MQ-9  Reaper.  The  multi- 
platform  radar  technology  insertion  program  (MP-RTIP)  is  the  active  electronically  scanned  aperture  radar  developed  for 
the  Global  Hawk  Block  40,  and  Lynx  is  the  radar  on  board  MQ-1  Predator  and  I-GNAT. 
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Summary  of  SCOPEM  Environment 


•  We  model  operational  effectiveness  with  a  series  of 
vignettes  to  capture  desired  capabilities 

*  Measure  effectiveness  in  the  vignette;  explore  variant 
space 

*  PAF’s  modeling  capability  is  structured  to  build  rich 
vignettes  out  of  simple  modules  of  code,  to  keep  it  all 
accessible 

•  Subsystem  modeling  allows  us  to  evaluate  sensor, 
platform,  and  environmental  factors  and  provides 
insight  into  the  mission-level  outcomes 


PAF  has  developed  a  suite  of  tools  and  a  mission-level  model  to  examine  the  operational  effec¬ 
tiveness  of  candidate  platforms  (e.g.,  manned  and  unmanned  planes,  satellites,  and  ground- 
based  systems)  to  perform  Air  Force  missions  of  interest.  We  model  operational  effectiveness 
with  a  series  of  vignettes  to  capture  desired  Air  Force  capabilities.  We  do  this  because  there  is 
not  a  one-to-one  mapping  of  capabilities  to  vignettes.  Within  each  vignette,  we  explore  varia¬ 
tions  in  outcomes  based  on  environmental  changes  and  system  modifications  (e.g.,  decreasing 
the  radar  signature  of  the  candidate  platform). 

PAF’s  modeling  capability  is  structured  to  build  a  rich  vignette  with  terrain,  multiple 
assets  operating  together,  varied  behaviors  for  platforms,  and  so  on.  The  modeling  occurs  in 
simple  modules  of  code  to  keep  the  SCOPEM  code  accessible.  Finally,  the  subsystem  modeling 
and  tools  that  PAF  has  developed  allow  us  to  evaluate  sensor,  platform,  and  environmental  fac¬ 
tors,  whether  independently  or  as  inputs  to  SCOPEM.  The  subsystem  tools  also  provide  insight 
into  the  mission-level  outcomes.  However,  mission-level  analysis  is  not  enough  to  inform  deci¬ 
sionmakers  as  to  the  necessary  force  structure  required  of  a  given  candidate.  An  examination  of 
candidate  performance  at  the  campaign  level  is  necessary  to  recommend  the  number  of  RPAs 
and  associated  costs  of  systems  for  the  overall  force  structure.  Before  we  discuss  our  campaign- 
level  modeling  capability  in  the  next  section,  we  conclude  the  mission-level  discussion  with  a 
look  at  possible  additional  tools  necessary  to  explore  the  full  range  of  mission  areas  for  RPAs. 
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Further  Tools  Will  Be  Developed  to  Meet 
Analytical  Needs 

•  Models  to  analyze  near-term  sensors  could  include 

-  directed  energy  weapons 

-  advanced  jammers 

-  additional  nonkinetic  attack  means 
- LADAR 

-  HSI 

-  enhanced  GMTI  model  to  account  for  STAP  gains 

-  geolocation  by  SIGINT 

-  CONOPS  for  sensor  cross-cuing 

-  instantaneous  data  rates 

*  Higher-level  trade  analysis  model  to  explore  overall 
force  structure  requirements  and  associated  cost 


Further  tools  can  be  developed  to  meet  the  analytical  needs  of  a  robust  assessment  of  candi¬ 
date  RPAs  in  Air  Force  missions.  Models  to  analyze  near-term  sensors,  for  example,  may  aid 
evaluation  of  future  mission  areas  using  laser  detection  and  ranging  (LADAR),  hyperspectral 
imagery  (HSI),  space-time  adaptive  processing  (STAP)  for  GMTI,  geolocation  by  SIGINT, 
CONOPS  development  for  cross-cuing  of  sensors,  and  instantaneous  data  rates. 

Once  mission-level  analysis  is  complete,  the  analyst  has  an  understanding  of  the  type  and 
number  of  RPAs  that  are  needed  to  perform  a  particular  mission  within  a  vignette.  But,  how 
does  this  transfer  to  how  many  platforms  are  needed  of  a  given  RPA  system?  A  higher-level 
trade  analysis  model  could  be  used  to  explore  the  overall  force  structure  requirements  and  asso¬ 
ciated  costs  of  a  given  candidate  system.  PAF  has  developed  a  model  to  aid  in  this  analysis,  too. 
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RPA  Evaluation  Methodology 


We  now  turn  our  discussion  to  campaign-level  questions  and  associated  analysis  that  help  us 
determine  the  necessary  force  structure  to  get  the  job  done.  For  example,  we  would  like  to 
know,  for  a  given  distribution  of  targets  over  an  area  of  responsibility  (AOR),  how  the  fleet  size 
affects  warfighting  outcomes  as  defined  by  a  given  set  of  MOEs.  MOEs  will  vary  depending 
on  the  commander’s  objectives  for  the  campaign.  For  nonshooting  scenarios,  the  MOE  may  be 
total  number  of  targets  imaged.  In  contrast,  for  a  shooting  war,  the  MOE  may  be  the  loss  ratio 
of  enemy  assets  (targets  damaged  or  destroyed)  to  blue’s  losses. 

We  saw  in  the  previous  slides  that  SCOPEM  captures  the  operational  effectiveness  of  a 
candidate  RPA  within  a  specified  vignette.  The  SCOPEM  mission-level  modeling  provides 
insights  into  the  different  capabilities  of  alternative  systems.  However,  to  understand  the  force 
structure  implications  of  employing  a  particular  RPA,  we  need  to  explore  RPA  employment  at 
the  campaign  level.  The  larger  campaign  scenario  will  fold  in  basing  locations  of  RPA  and  the 
frequency  with  which  missions  occur  (i.e.,  ground  truth  on  target  distributions  in  time  and 
space).  The  following  slides  present  PAF’s  FSE  model,  which  is  equipped  to  help  address  these 
issues. 
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FSE  Is  a  Fleet  Sizing  Model  Based  on  Aircraft 
and  Sensor  Performance 


SEAS/SCOPEM 
Outputs:  For  each  alternative:  For 
a  given  search  area,  terrain, 
weather,  and  target  distribution,  a 
database  of  possible  outcomes  in 
terms  of  results  and  timing 
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MOE  Outputs:  For  a  given  distribution  of  targets 
over  an  AOR,  what  is  the  level  of  overall  MOE 
achieved  as  a  function  of  fleet  size? 


FSE  models  each  RPA  alternative  as  a  fleet  of  RPAs  and  measures  the  effectiveness  of  that  fleet 
against  various  individual  MOEs.  FSE  includes  four  primary  modeling  components:  an  air¬ 
craft  flight  performance  model,  a  FSE/SCOPEM  interface  database,  a  campaign-level  scenario 
model,  and  an  optimized  RPA  placement  model.  The  overarching  objective  is  to  character¬ 
ize  the  effectiveness  of  various  aircraft  and  sensor  platforms  as  a  function  of  fleet  size  across 
a  theater.  This  effectiveness  measure  provides  the  effectiveness  piece  of  the  cost-effectiveness 
analysis — in  other  words,  the  “bang”  in  “bang  for  the  buck.”  FSE  currently  models  mul¬ 
tiple  types  of  ISR/strike  demands,  including  popup  demands,  HVT  tracking,  and  combat  air 
support. 

The  campaign-level  scenario  model  and  the  optimized  RPA  placement  model  run  concur¬ 
rently  and  draw  on  the  flight  performance  model  and  the  FSE/SCOPEM  interface  database. 
Specifically,  both  the  optimized  RPA  placement  model  and  the  campaign-level  scenario  model 
step  through  time  modeling  the  global  scenario  and  RPA  positions  and  recording  the  results 
of  RPA  engagements.  At  every  time  step  in  the  model,  each  RPA  is  repositioned  based  on  its 
speed  and  direction,  and  the  fuel  onboard  the  RPA  is  decremented.  The  speed  and  fuel  burn 
are  driven  by  the  flight  performance  model,  whereas  the  flight  direction  is  determined  by  the 
optimized  RPA  placement  model.  The  campaign-level  scenario  model  uses  the  global  scenario 
probabilities  to  determine  whether  and  where  a  demand  occurs  at  every  time  step.  The  opti¬ 
mized  RPA  placement  model,  if  RPAs  are  available,  will  send  RPAs  to  meet  the  demand  and 
reposition  all  other  RPAs  in  anticipation  of  additional  future  demands.  Once  an  RPA  arrives 
to  fulfill  a  demand,  an  appropriate  random  draw  is  made  from  the  FSE/SCOPEM  interface 
database  to  determine  the  outcome  of  the  RPA  fulfilling  the  demand. 
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FSE  Flight  Performance  Model  Flies  RPA 
Through  Each  Phase  of  the  Mission 

•  RPA  characterized  by  several  key  inputs 

-  Weights:  OEW,  MGTOW,  payload,  fuel 

-  Speed  envelope:  function  of  weight,  altitude 

-  Altitude  envelope:  function  of  weight,  speed 

-  Specific  range:  function  of  weight,  speed,  and  altitude 

-  Takeoff  and  landing  performance:  function  of  weight,  field 
elevation,  and  field  temperature 

*  RPA  is  modeled  through  all  phases  of  flight,  both  in  the  air 
and  on  the  ground 

-  MC  rate 

-  Ground  time:  load/unload  time,  refuel  time 

-  Start,  taxi,  takeoff:  time  spent  and  fuel  burned 

-  Climb:  time  spent  and  fuel  burned 

-  Cruise/dash:  time  spent  and  fuel  burned  based  on  specific 
range  and  speed 

-  Approach  and  land:  time  spent  and  fuel  burned 


The  FSE  aircraft  flight  performance  model  uses  specific  range  data  (distance  traveled  per  unit 
fuel  weight)  as  a  function  of  RPA  weight,  altitude,  and  speed  in  order  to  compute  fuel  burn. 
The  model  also  incorporates  RPA  data  on  maximum  speed  and  altitude  capability  as  a  func¬ 
tion  of  weight  and  altitude,  as  well  as  takeoff  and  landing  performance  as  a  function  of  aircraft 
weight,  airfield  elevation,  and  airfield  temperature. 

Aircraft  endurance,  flight  envelope,  and  response  time  are  computed  by  integrating  fuel 
burn  and  speed  throughout  the  flight.  Each  RPA  is  subject  to  standard  fuel  reserve  require¬ 
ments.  The  FSE  flight  model  accurately  allows  for  the  trade-off  between  speed  and  endurance 
to  be  optimized — for  example,  the  overall  FSE  model  can  determine  the  circumstances  under 
which  a  high-speed  dash  is  appropriate  given  that  a  high-speed  dash  reduces  endurance.  The 
flight  model  also  includes  fuel  burn  during  start,  taxi,  takeoff,  climb,  approach,  and  land. 
In  addition  to  modeling  fuel  burn,  the  FSE  aircraft  flight  performance  model  also  computes 
aircraft  ground  times  and  flight  times.  Specifically,  this  translates  to  the  percentage  of  time  a 
single  RPA  is  on  station  and  available  to  fulfill  demands.  Finally,  the  FSE  aircraft  flight  per¬ 
formance  model  includes  details  on  aircraft  takeoff  and  landing  distances  as  a  function  of  field 
elevation  and  temperature.  To  the  extent  the  RPA  is  limited  by  the  field  conditions,  the  model 
will  reduce  aircraft  payload  or  fuel  to  accommodate  the  given  field  conditions. 
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FSE/SCOPEM  Interface  Database:  Integrates 
SCOPEM  Results  into  Campaign-Level  Scenario 

•  Sensor  performance  is  modeled  through  Monte  Carlo 
integration  with  SCOPEM 

-  A  very  large  number  of  SCOPEM  runs  are  recorded 
in  a  database,  and  FSE  randomly  draws  from  runs 
with  characteristics  similar  to  those  of  the 
campaign-level  scenario 

*  FSE  matches  characteristics  between  the  global 
scenario  and  SCOPEM  runs 

-  These  characteristics  include  time  of  day,  weather, 
terrain,  and  target  type,  as  well  as  other  scenario- 
specific  characteristics 


The  FSE  model  does  not  include  any  inherent  sensor  modeling.  For  this,  the  FSE  model  relies 
on  an  FSE/SCOPEM  interface  database  using  a  Monte  Carlo  approach.  This  database  includes 
a  very  large  number  of  runs;  each  run  includes  information  on  the  conditions  under  which  the 
run  occurred  (for  example,  rime  of  day,  weather,  terrain,  target  type)  and  other  parameters. 
The  FSE  model,  based  on  the  demand  defined  by  the  global  scenarios,  will  then  randomly 
draw  a  SCOPEM  run  from  the  database  with  conditions  that  match  the  conditions  specified 
in  the  global  scenario.  This  random  draw  is  important  because  SCOPEM  data  are  inherently 
stochastic  because,  even  with  a  fixed  set  of  scenario  parameters,  there  is  a  probability  of  success. 
The  database  pull  will  then  provide  the  FSE  model  with  information,  such  as  the  outcome  of 
the  event  and  the  timing  of  the  event.  As  a  simple  example,  the  global  scenario  may  have  an 
FIVT  cue  at  an  airport  and  require  that  the  HVT  be  tracked.  If  this  occurs  at  noon  on  a  clear 
day  on  flat,  open  terrain,  the  FSE  model  would  pull  an  HVT  track  run  from  the  interface 
database  that  also  occurred  around  noon  on  a  clear  day  on  flat,  open  terrain.  This  pull  would 
then  specify,  for  example,  that  the  HVT  was  identified  after  some  minutes  of  arriving  over¬ 
head  and  that  the  HVT  was  successfully  tracked  in  a  vehicle  for  a  given  amount  of  time  before 
the  target  was  lost.  The  FSE  model  would  then  use  the  results  of  this  data  pull  to  fly  the  RPA 
through  the  mission  and  record  the  outcome  for  computation  of  the  MOEs.  This  methodology 
allows  FSE  to  fully  capture  the  subtleties  of  sensor  performance  and  stochastics  by  leveraging 
the  power  of  SCOPEM. 
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FSE  Campaign-Level  Scenario  Model:  Multiple  Scenarios 
Across  a  Theater  to  Compute  Fleet  Size  and  Effectiveness 

•  A  theater  is  modeled  by  probabilities  of  certain  events 
occurring;  these  events 
include  popup  demands, 

CAS,  and  HVT  tracking 

•  RPAs  are  continuously 
repositioning  themselves 
based  on  changing 
potentials 

•  Threat  probabilities  may 
also  be  included  and  aircraft 
behavior  modified  to  account 
for  additional  risk 


The  FSE  campaign-level  scenario  is  described  by  probabilities  of  various  events  occurring  over 
space  and  time.  The  model  allows  for  a  ground-truth  distribution  and  an  anticipated  distribu¬ 
tion.  In  this  slide,  the  color  represents  the  probability  of  a  popup  demand  occurring.  Dark  red 
indicates  the  areas  where  popup  demands  are  most  likely  to  occur;  dark  blue  indicates  where 
these  demands  are  least  likely  to  occur.  In  addition,  there  are  four  small  black  dots,  which  rep¬ 
resent  locations  where  FIVT  cues  may  occur. 

RPA  behavior  is  modeled  using  a  “potential  method.”  This  method  uses  a  potential  similar 
to  gravitational  potential.  In  particular,  demands  are  treated  as  attractive,  while  the  RPAs  exert 
a  repulsive  force  on  each  other.  This  means  that  RPAs  will  position  themselves  near  areas  of 
high-demand  potential,  in  order  to  maximize  their  odds  of  successfully  meeting  any  demand 
that  occurs,  but,  because  of  the  mutual  repulsive  force  between  them,  they  will  not  all  cluster 
in  one  location.  This  formulation  encourages  the  RPAs  to  spread  themselves  over  the  battle 
space.  With  this  potential  model,  the  first  RPA  will  be  placed  at  the  point  of  highest  potential; 
as  additional  RPAs  are  added,  they  will  reposition  themselves  to  better  cover  the  battle  space 
while  still  providing  sufficient  coverage  to  the  highest-demand  regions.13  The  model  continu¬ 
ously  repositions  RPAs;  therefore,  if  an  RPA  returns  to  base  to  refuel,  a  new  RPA  arrives,  or 
the  RPA  is  called  off  to  fulfill  a  demand,  all  the  other  RPAs  will  reposition  themselves  to  again 
maximize  the  chance  of  meeting  anticipated  future  demands. 


13  The  purpose  of  a  potential  model  is  to  simulate  the  approximate  outcome  of  an  intelligent  employment  scheme  without 
actually  having  to  model  artificial  intelligence  within  the  program.  We  do  not  propose  that  RPAs  would  actually  distribute 
themselves  via  this  method,  nor  that  this  method  would  be  used  to  calculate  their  positions  by  some  central  authority. 
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The  potential  method  has  several  parameters,  which  ultimately  control  the  RPA  behavior. 
These  parameters  are  found  by  performing  an  optimization14 — that  is  to  say,  the  parameters 
are  set  in  order  to  maximize  the  overall  MOE.  These  parameters  include  the  relative  attrac¬ 
tiveness  of  various  demand  types,  the  distance  over  which  these  demands  are  attractive,  and 
the  repulsiveness  of  the  RPAs  from  each  other.  As  an  addition  to  FSE,  the  threat  environment 
may  be  described  as  a  repulsive  potential,  which  would  act  to  push  RPAs  away  from  regions  of 
high  threat.  The  amount  of  repulsion  would  be  an  additional  parameter  used  to  describe  RPA 
behavior. 


14  A  Nelder-Mead  search  method  is  used  to  maximize  the  MOE  and  to  find  the  unconstrained  parameters.  This  search 
method  does  not  rely  on  local  gradients,  which  is  important  because  the  MOE  is  not  a  smooth  function  of  the  parameters. 
The  lack  of  smoothness  results  from  the  fact  that  a  very  small  change  in  the  parameters  often  results  in  no  change  to  the 
MOE;  when  a  change  does  occur,  it  can  be  very  significant.  However,  convergence  to  a  global  maximum  is  not  guaranteed 
because  the  MOE  is  not  a  concave  function  of  the  parameters.  In  order  to  improve  confidence,  many  different  starting 
points  are  used  in  the  optimization  routine. 
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FSE-Optimized  RPA  Placement  Model:  Parameters 
Depend  on  the  Candidate  Platform  Characteristics 

and  the  Fleet  Size 


•  Aircraft  characteristics,  such  as  speed  and  sensor 
performance,  will  lead  to  different  RPA  behaviors 


RPA  behavior  is  optimized 
to  net  the  greatest 
percentage  of  weighted 
demands  met 
Optimized  behavior 
attracts  RPAs  to  areas  of 
high-demand  potential 
but  repels  them  from  one 
another 


*  Behavior,  and  ultimately  effectiveness,  is  a  function 
of  fleet  size 


The  FSE-optimized  RPA  placement  model  finds  the  optimum  parameters  for  the  potential 
model,  which  results  in  each  candidate  RPA  maximizing  its  overall  MOE.  Each  candidate 
platform  will  have  different  optimum  parameters  and  will  therefore  behave  differently;  how¬ 
ever,  because  of  the  optimization,  each  alternative  will  behave  in  a  way  that  is  similarly  opti¬ 
mal.  The  RPA  potential  parameters  will  also  depend  on  fleet  size.  Small  fleets  will  tend  to  focus 
on  only  the  highest-demand  regions,  and  each  RPA  will  position  itself  far  from  the  others.  On 
the  other  hand,  large  fleets  will  focus  on  all  demand  areas,  and  the  RPAs  will  cluster  closer 
together. 

As  an  example,  consider  a  fast  RPA  versus  a  slow  RPA.  The  fast  RPA  would  be  able  to 
cover  a  much  larger  area  in  a  fixed  amount  of  time  than  a  slow  RPA;  therefore,  the  optimiza¬ 
tion  routine  would  assign  a  higher  repulsiveness  to  the  fast  RPAs  than  the  slow  ones.  This  is 
because  slow  RPAs  have  a  limited  area  that  they  can  affect,  so  they  need  to  be  spaced  closer 
together  than  do  fast  RPAs.  All  else  being  equal,  this  optimization  would  lead  to  the  conclu¬ 
sion  that  a  larger  number  of  slow  RPAs  is  required  to  produce  the  same  level  of  effectiveness 
as  a  smaller  number  of  fast  RPAs.  This  is  a  critical  finding  because,  once  the  precise  number  is 
computed,  it  is  possible  to  determine  which  platform  is  the  most  cost-effective.  In  this  example, 
this  would  mean  determining  whether  it  is  more  cost-effective  to  have  a  large  number  of  slow 
RPAs  or  a  small  number  of  fast  RPAs.  If  the  fast  RPA  is  more  expensive  than  the  slow  RPA, 
the  answer  is  not  immediately  obvious,  and  FSE  would  provide  the  detailed  analysis  required 
to  address  this  issue. 

The  FSE-optimized  RPA  placement  model  uses  the  anticipated  demand,  which  is  the  esti¬ 
mate  of  the  ground-truth  demand  from  the  campaign-level  scenario,  to  position  the  RPAs  in  a 
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fashion  that  results  in  them  being  most  likely  to  Fulfill  these  anticipated  demands.  Specifically, 
the  optimized  parameters  will  result  in  the  RPAs  placing  themselves  close  to  regions  where 
demands  are  anticipated  to  occur  and  farther  from  regions  where  demands  are  not  anticipated 
to  occur.  Given  that  the  anticipated  demand  is  in  general  different  from  the  ground-truth 
demand,  the  RPAs  will  position  themselves  to  maximize  their  chance  of  meeting  the  antici¬ 
pated  demands,  which  means  that  they  will  not  be  optimally  placed  for  the  actual  ground- 
truth  demand.  This  is  correct  modeling  because  the  ground  truth  would,  in  general,  not  be 
known  a  priori  to  RPA  operators.  The  distinction  between  ground-truth  demand  and  antici¬ 
pated  demand  is  important.  For  example,  to  the  extent  that  the  ground-truth  demand  is  not 
known  accurately,  a  fast  RPA  can  more  rapidly  adjust  its  position  based  on  actual  events. 

If  a  threat  environment  is  included  in  the  global  scenario,  then  the  repulsive  potential 
would  also  be  optimized  for  each  RPA  platform,  such  that  the  optimum  balance  between 
threat  level  and  demand  level  could  be  found.  For  example,  imagine  two  scenarios,  an  area 
of  high-demand  potential  but  medium  threat  and  an  area  of  medium-demand  potential  but 
high  threat.  In  this  case,  FSE  may  find  that  the  optimum  solution  is  that  neither  of  these 
areas  should  be  entered,  that  only  the  high-demand  and  medium-threat  environment  should 
be  entered  or  both  should  be  entered.  In  addition,  this  determination  would  be  a  function  of 
fleet  size.  For  very  small  fleets,  FSE  optimization  would  show  a  risk  aversion,  but,  as  the  fleet 
size  was  increased,  this  risk  aversion  would  decrease.  Furthermore,  the  platforms’  survivability 
would  be  crucial  in  determining  the  repulsiveness  of  the  threat  environment.  A  highly  surviv- 
able  platform  would  be  less  repelled  from  a  threat  environment  than  would  a  less  survivable 
platform. 

The  optimized  placement  model  is  a  critical  component  of  FSE  for  several  reasons.  The 
model  allows  for  any  fleet  size  to  be  analyzed.  The  model  allows  for  RPA  alternatives  with  very 
different  characteristics  to  be  utilized  in  a  manner  that  maximizes  their  utility.  The  model  also 
provides  a  systematic  way  to  position  RPAs  when  the  AOR  is  larger  than  can  be  fully  covered 
by  a  given  fleet  of  RPAs,  i.e.,  the  model  is  essential  when  defined  orbit  locations  are  not  suf¬ 
ficient  to  meet  all  demands.  Finally,  the  model  allows  for  dynamic  allocation  to  respond  to 
changes  in  demand  and  to  changes  in  the  number  of  RPAs  on  station. 

The  slide  shows  how  RPAs  would  position  themselves  using  the  optimization  method 
discussed.  Again,  the  red  areas  represent  regions  of  high  demand,  and  blue  represents  regions 
of  low  demand.  In  this  image,  it  is  clear  that  the  RPAs  are  clustering  around  the  high-demand 
regions  but  then  spacing  themselves  out  over  the  lower-demand  regions.  This  is  exactly  the 
behavior  one  would  expect;  specifically,  one  would  want  to  first  fully  cover  the  high-demand 
regions  and  then  space  out  the  remaining  RPAs  over  the  lower-demand  regions. 

This  image  does  not  depict  the  dynamic  nature  by  which  the  RPAs  are  positioned.  To 
illustrate  this,  imagine  that  the  single  RPA  placed  at  the  upper  red  region  was  called  off  to  ful¬ 
fill  a  demand;  this  would  mean  that  there  would  be  no  RPAs  immediately  nearby  to  fulfill  a 
possible  future  demand  in  the  area  of  that  upper  red  region.  Because  the  RPA  that  was  called 
off  to  fulfill  a  demand  would  no  longer  be  providing  a  repulsive  force  to  RPAs  around  it,  there 
would  be  a  potential  gradient  moving  the  nearby  RPAs  into  this  gap.  As  these  RPAs  move  in 
to  fill  the  gap,  they  would  also  be  less  repulsive  to  the  other  RPAs  around  them,  and  this  effect 
would  cascade  as  a  series  of  RPAs  all  repositioned  themselves.  In  effect,  the  slide  shows  the 
steady-state  equilibrium  for  n  RPAs.  If  one  RPA  is  called  to  fulfill  a  demand,  the  remaining 
RPAs  would  reposition  themselves  into  the  steady-state  equilibrium  for  n  -  1  RPAs. 
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FSE  Output:  Effectiveness  as  a  Function  of 

Number  of  RPAs 

•  For  a  given  level  of  effectiveness,  the  number 
of  RPAs  required  can  be  calculated 

•  For  a  fixed  budget  (number  of  RPAs 
acquired),  the  effectiveness  can  be  calculated 


A  cost-effectiveness  analysis  can  be  thought  of  in  one  of  two  ways.  First,  for  a  given  budget 
level,  what  RPA  platform  will  be  the  most  effective?  Or  second,  for  a  given  level  of  effectiveness, 
what  RPA  platform  fleet  is  the  cheapest?  FSE  provides  the  effectiveness  piece  of  the  answer  to 
this  question,  which  can  then  be  combined  with  cost-estimating  to  determine  the  most  cost- 
effective  RPA  platform.  For  example,  consider  the  notional  chart  on  this  slide.  Effectiveness 
is  plotted  as  a  function  of  number  of  RPAs.  Consistently,  the  blue  platform  outperforms  the 
red  platform;  this  means  that  the  blue  platform  is  more  effective.  The  simplest  case  is  that  in 
which  the  blue  RPA  is  cheaper  than  the  red  RPA,  which  would  mean  that  the  blue  RPA  would 
be  more  cost-effective.  Ffowever,  it  is  often  the  case  that  the  most  effective  platform  is  not  the 
cheapest.  When  the  most  effective  RPA  is  not  also  the  cheapest  RPA,  a  full  cost-effectiveness 
calculation  is  required. 

In  this  case,  the  relative  cost  of  a  blue  versus  red  fleet  could  be  calculated.  For  illustrative 
purposes,  we  will  choose  a  0.6,  or  60-percent  effectiveness  level.  This  means  that,  of  all  the 
demands  that  were  simulated,  60  percent  of  them  were  successfully  fulfilled.  Using  this  effec¬ 
tiveness  requirement,  the  charts  show  that  it  would  require  four  blue  RPAs  or  six  red  RPAs. 
Therefore,  if  the  relative  cost  of  the  blue  RPA  to  red  RPA  is  less  than  1.5,  the  blue  RPA  would 
be  the  most  cost-effective;  however,  if  the  relative  cost  of  the  blue  RPA  to  the  red  RPA  is  greater 
than  1.5,  the  red  RPA  would  be  the  most  cost-effective. 
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RPA  Evaluation  Methodology 


This  documented  briefing  summarizes  models  and  methods  within  PAF’s  toolbox  for  analyz¬ 
ing  many  of  the  factors  that  affect  the  operational  effectiveness  of  RPAs  and  other  candidates 
(including  space  systems  and  ground-based  assets)  under  consideration  to  bridge  potential  gaps 
in  Air  Force  capabilities.  We  presented  the  individual  tools  developed  at  RAND  within  the 
past  five  years,  models  created  in  FY  2010  to  round  out  our  capabilities,  and  how  existing  Air 
Force  models  were  integrated  into  our  operational  effectiveness  analysis  using  SCOPEM  and 
into  our  force  structure  effectiveness  analysis  using  FSE. 

The  suite  includes  tools  to  analyze  specific  aspects  of  platforms,  sensor  performance 
against  various  targets,  weapon  effects,  environmental  factors,  platform  survivability,  weapon 
employment,  computational  processing  of  data,  and  exploitation  of  sensor  products.  These 
individual  tools  contribute  to  the  mission-level  analysis  performed  with  SCOPEM.  SCOPEM 
is  structured  to  build  a  rich  vignette,  including  terrain  effects,  multiple  assets  operating 
together,  varied  behaviors  for  platforms,  and  other  features  that  simulate  complex  operational 
environments.  The  modeling  occurs  in  simple  modules  of  code,  which  provide  insight  into  the 
factors  that  drive  mission-level  outcomes.  MOE  examples  derived  from  SCOPEM  modules 
include  detection  of  a  target  for  a  sensor,  LOS  obscuration  from  terrain,  and  Pk  from  weapon 
employment.  This  level  of  detail  is  essential  to  building  an  effectiveness  matrix,  which  not  only 
identifies  effective  platforms  and  CONOPS  but  defines  the  range  of  conditions  under  which 
platforms  either  succeed  or  fail  at  a  given  mission. 

Since  it  was  initiated  in  2005,  SCOPEM  has  supported  a  variety  of  studies,  including  the 
following: 
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•  “Non-Traditional  Intelligence,  Surveillance,  and  Reconnaissance”  (FY  2008-2009;  Jody 
Jacobs,  Bart  Bennett;  sponsored  by  USAF/A2) 

•  “Satisfying  the  Demand  for  Surveillance  and  Reconnaissance  in  the  European  and  Afri¬ 
can  Theaters”  (FY  2008;  Carl  Rhodes;  sponsored  by  USAF/A2  and  USAFE/A2) 

•  “The  Role  of  Global  Hawk  in  Maritime  Surveillance”  (FY  2006—2007;  Sherrill  Lingel, 
Carl  Rhodes;  sponsored  by  PACAF/A2,  AF/A2,  ACC/A2,  and  ACC/A8) 

•  “Tasking  and  Employing  USAF  Intelligence,  Surveillance,  and  Reconnaissance  Assets  to 
Support  Effects-Based  Operations”  (FY  2005-2006;  Sherrill  Lingel,  Carl  Rhodes;  spon¬ 
sored  by  PACAF/A2). 

One  must  also  understand  force  structure  implications  of  employing  a  particular  RPA. 
To  do  so,  we  explore  RPA  employment  at  the  campaign  level.  For  example,  we  would  like  to 
know,  for  a  given  distribution  of  targets  over  an  AOR,  the  effectiveness  level  as  a  function  of 
fleet  size.  A  campaign  scenario  should  include  mission-level  insights,  as  well  as  broader  con¬ 
siderations,  such  as  basing  locations  of  RPAs  and  the  demand  frequency  of  mission  occur¬ 
rence  (i.e.,  ground  truth  on  target  distributions  in  time  and  space).  PAF  has  developed  the 
FSE  model  to  perform  this  campaign  analysis.  The  campaign  look  afforded  by  FSE  results  in 
a  required  force  size  under  varying  effectiveness  levels.  The  previously  mentioned  individual 
tools  and  the  mission-level  outcomes  from  SCOPEM  inform  the  campaign  model,  FSE.  Last, 
when  the  force  structure  evaluation  is  coupled  with  cost  analysis,  a  cost-effectiveness  examina¬ 
tion  of  the  candidate  systems  is  created. 

Taken  together,  this  suite  of  models  and  tools  can  help  the  Air  Force  explore  the  most 
cost-effective  ways  to  take  advantage  of  the  unique  capabilities  of  RPAs  in  the  future.  PAF  is 
now  using  the  SCOPEM  model  to  study  a  set  of  roles  and  missions  for  next-generation  RPAs. 


APPENDIX 


Additional  Detail  on  Selected  Models 


This  appendix  provides  additional  details  about  the  radar  models  (GMTI  and  SAR),  as  well  as 
the  survivability  and  air  defense  engagement  models  described  in  the  body  of  the  documented 
briefing. 


Additional  Radar  Details 

Ground  Moving  Target  Indicator 

GMTI  uses  two  or  more  pulses  to  coherently  filter  out  stationary  ground  clutter,  thereby  pro¬ 
viding  detection  gain  against  stationary  clutter  when  targets  are  moving.  In  practice,  ground 
clutter  is  spread  out  in  velocity  because  of  its  own  motion  (e.g.,  grass  moving  in  the  wind)  and 
because  of  the  spread  in  platform  velocity  across  the  beam.  Thus,  the  amount  of  gain  depends 
on  the  target’s  Doppler  velocity  with  respect  to  the  apparent  width  of  the  clutter  spectrum. 
If  a  target  is  at  a  velocity  that  exceeds  the  spread  in  clutter  velocities,  we  say  that  the  target  is 
detected  in  exoclutter — that  is,  against  noise  alone.  Otherwise,  it  is  detected  in  endoclutter — 
that  is,  against  both  clutter  and  noise.  To  model  the  endoclutter  that  limits  detection  at  slow 
target  speeds,  we  have  developed  a  model  that  accounts  for  both  the  structure  of  the  Doppler 
filter  and  the  clutter  Doppler  spectrum.  From  the  radar  parameters,  we  use  the  radar  range 
equation  to  compute  power  received  at  the  radar  from  a  signal  and  from  the  clutter  at  the  sig¬ 
nal’s  Doppler  velocity.  These  depend  on  the  signal  and  clutter  RCSs.  We  have  implemented 
clutter  cross-sections  for  a  wide  range  of  environments,  from  meadows  to  asphalt,  and  have 
parameterized  them  as  a  function  of  grazing  angle  (Long,  2001).  (This  grazing  angle  depen¬ 
dence  accounts  for  the  behavior  of  SCNR  on  slant  range.)  SCNR  generally  will  be  smallest  at 
very  small  or  very  large  slant  range.  This  is  because,  at  close  slant  range,  clutter  RCS  can  be 
very  large  due  to  nearly  specular  backscatter  and,  at  very  long  slant  range,  the  signal  energy 
becomes  small  compared  to  the  noise  and  clutter.  Under  appropriate  simplifying  assumptions, 
PD  is  related  to  SCNR  and  PFA  by  a  simple  expression. 

As  a  test  of  the  model,  we  compared  the  minimum  detectable  RCS  that  we  derived  with 
a  value  reported  for  the  Lynx  radar  at  25  km  at  an  altitude  of  3,048  m  (10,000  ft).  The  Lynx 
GMTI  specification  states  that  it  can  detect  a  3  m-per-second  target  of  10  dBsm  at  25  km  in 
-10  dB  clutter  (“Lynx,”  2009).  Our  model  predicts  that  the  Lynx  radar  with  32-pulse  coherent 
processing  can  achieve  this  performance  at  platform  speeds  typical  of  a  Predator  MQ-1  and 
that  128-pulse  processing  could  meet  this  performance  from  platforms  of  much  higher  speed. 
(At  present,  our  model  does  not  account  for  improvements  from  STAP,  which  is  likely  to  be 
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available  on  any  radar  that  would  be  fielded  on  a  future  RPA  that  uses  active  electronically 
scanned  array  antennas.) 

The  GMTI  model  is  implemented  in  SCOPEM  as  a  table  of  clutter- to -noise  ratio  (CNR) 
values  for  a  range  of  target  Dopplers  and  platform  cross-range  speeds.  This  table  is  supple¬ 
mented  with  target  SNR  at  a  set  of  reference  values.  The  SNR  and  CNR  are  readily  scaled 
from  the  reference  values  to  other  sensor  altitudes,  resolutions,  or  target  RCSs.  From  these,  the 
SCNR  and  the  resulting  PD  are  easily  computed.  The  CNR  table  needs  to  be  recomputed  for 
each  set  of  radar  parameters. 

GMTI  modes  often  are  used  in  conjunction  with  a  tracking  algorithm  to  track  moving 
targets.  Although  GMTI  modes  can  have  good  range  resolution,  they  generally  have  poor 
cross-range  resolution,  dictated  only  by  the  beam  width.  Thus,  range  and  cross-range  localiza¬ 
tions  are  highly  asymmetric,  and  tracking  performance  will  depend  on  the  direction  of  the 
target’s  velocity  with  respect  to  the  range  direction.  In  addition  to  the  sensor’s  ability  to  localize 
the  target,  tracking  depends  on  the  potential  to  confuse  the  track  of  one  target  with  another 
as  each  target  moves  between  revisits.  This  potential  for  track  confusion  depends  on  how  pre¬ 
dictable  future  target  locations  are  (i.e.,  on  a  target  motion  model)  and  on  the  target  density. 

In  tracking  algorithms,  single  or  multiple  target  measurements  are  used  to  measure 
things,  such  as  target  position,  velocity,  or  acceleration.  These  measurements  define  the  “state” 
vector  of  the  target.  Usually,  tracking  algorithms  attempt  to  predict  future  state  vectors  and  to 
combine  these  predictions  with  future  measurements  to  arrive  at  a  more  accurate  determina¬ 
tion  of  actual  target  location  (e.g.,  the  Kalman  filter).  Future  state  vectors  often  are  predicted 
under  the  assumption  that  the  target  is  not  accelerating.  Instead,  the  possibility  that  a  target 
accelerates  represents  a  source  of  uncertainty  in  the  tracking  model,  and  we  make  this  assump¬ 
tion  here.  It  is  not  our  intention  to  define  or  model  specific  algorithms  that  would  be  used  in 
a  real  tracking  processor  (e.g.,  Multi-Hypothesis  Trackers  or  probabilistic  data  association  fil¬ 
ters)  but  to  capture,  in  relatively  simple  terms,  nominal  performance  and  explicitly  exhibit  the 
dependence  on  essential  sensor  parameters,  e.g.,  resolution  or  revisit  rate;  on  target  parameters, 
e.g.,  speed,  direction,  and  signal  strength;  and  on  environment,  e.g.,  clutter  levels  and  the  den¬ 
sity  of  confusion  targets. 

To  accomplish  this,  we  have  implemented  a  tracking  model  by  Mori,  Chang,  and  Chong, 
1992,  and  have  focused  on  results  that  correspond  to  a  two-element  state  vector  that  stores 
target  position.  This  model  is  made  analytically  tractable  with  the  aid  of  some  simplifying 
assumptions: 

•  The  probability  of  any  measurement  can  be  represented  by  a  multivariate  Gaussian 
distribution. 

•  Covariance  matrices  describing  measurement  uncertainties  are  the  same  for  all  targets. 

•  The  envelope  of  possible  measurements  extends  far  beyond  the  uncertainty  ellipse  defined 
by  the  covariance  matrix. 

•  Target  state  vectors  are  distributed  uniformly  over  the  measurement  envelope. 

•  When  many  targets  are  present  within  the  measurement  envelope,  the  probability  of  find¬ 
ing  even  one  target  within  the  uncertainty  ellipse  is  small  (i.e.,  targets  have  low  density 
with  respect  to  measurement  accuracy). 
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As  a  consequence  of  these  assumptions,  the  model  provides  meaningful  results  only  when 
the  probability  of  maintaining  track  between  revisits  is  high.  Where  probabilities  become  less 
than  0.5,  the  assumptions  of  the  model  are  violated. 

We  have  implemented  this  model  in  a  form  sufficiently  general  to  have  potential  appli¬ 
cation  to  tracking  by  other  sensors  as  well,  although,  to  date,  we  have  modeled  only  GMTI 
tracking  using  the  two-element  state  vector.  The  results  can  be  very  sensitive  to  revisit  interval, 
with  details  depending  on  the  sensor  accuracy,  motion  model,  and  density  of  targets.  At  a  suf¬ 
ficiently  long  revisit  interval,  the  probability  of  maintaining  track  will  become  small  as  long  as 
more  than  one  target  is  present  and  future  target  positions  have  uncertainty. 

The  model  has  been  implemented  in  SCOPEM  as  tables  of  probability  of  maintaining 
track  for  a  range  of  revisit  intervals,  target  orientations,  and  SCNRs  at  a  set  of  reference  values 
that  includes  the  target  density.  Values  are  readily  scaled  to  different  target  densities.  These 
tables  need  to  be  recomputed  for  each  set  of  radar  specifications.  In  many  cases,  for  reasons  of 
computational  necessity,  SCOPEM  time  steps  will  be  longer  than  the  revisit  interval  required 
to  maintain  track.  This  is  handled  easily  by  scaling  the  probability  for  any  revisit  interval  to  the 
SCOPEM  time  step,  and  this  will  yield  sensible  results  as  long  as  such  scaling  does  not  require 
a  computation  in  the  regime  where  the  model  is  inapplicable. 

Synthetic  Aperture  Radar 

Our  target  modeling  takes  into  account  how  the  three-dimensional  shape  of  the  target  can 
lead  to  effective  resolutions  that  differ  from  the  range/cross-range  resolution  in  the  slant  plane. 
Three  effects  are  of  potential  importance.  First,  at  low  grazing  angles,  horizontal  surfaces  are 
maximally  resolved  in  range,  whereas  vertical  surfaces  are  poorly  resolved,  appearing  in  only 
one  range  bin  at  the  lowest  grazing  angles.  The  mean  GSD  can  be  obtained  by  weighting  hori¬ 
zontal  and  vertical  surfaces  by  their  respective  areas.  Second,  portions  of  a  target  could  be  in 
shadow  at  sufficiently  low  grazing  angles,  making  less  of  the  well-resolved  horizontal  surfaces 
visible.  Third,  target  RCS  could  be  enhanced  at  low  grazing  angles  that  favor  specular-like 
reflections,  increasing  the  relative  contribution  to  total  RCS  of  the  poorly  resolved  vertical  sur¬ 
faces.  In  particular,  the  RCS  of  most  targets  is  dominated  by  the  discrete  returns  from  corners, 
edges,  and  interfaces,  and  we  approximate  targets  as  being  composed  of  dihedral  and  trihe¬ 
dral  reflectors  that  tend  to  be  oriented  in  the  horizontal  and  vertical  directions.  Each  of  these 
three  effects  tends  to  increase  the  relative  importance  of  vertical  surfaces  at  low  grazing  angles, 
resulting  in  a  reduction  in  effective  resolution. 

Modeling  the  first  effect  of  resolution  on  horizontal  versus  vertical  surfaces  is  a  simple 
matter  of  geometry,  in  which  the  slant  range  resolution  is  projected  onto  either  horizontal  or 
vertical  planes.  The  shadowing  effect  clearly  depends  on  how  target  shapes  deviate  from  simple, 
flat  surfaces.  For  example,  a  car  with  a  centerline  bulge  along  the  length  of  the  roof  would  have 
half  of  its  horizontal  surfaces  in  shadow  at  very  low  grazing  angles.  A  more  complex  target, 
such  as  a  tank,  could  have  even  more  of  its  parts  in  shadow  (e.g.,  by  the  turret),  and  shadowing 
is  likely  to  become  a  factor  at  higher  grazing  angles  than  for  a  car  (perhaps  at  angles  as  high 
as  several  tens  of  degrees).  High-fidelity  modeling  would  require  computer-aided  design  repre¬ 
sentations  of  vehicles,  something  far  out  of  scope  for  this  effort.  Instead,  we  adopted  a  simple 
model  in  which  the  fraction  of  horizontal  surfaces  visible  is  represented  by  a  linear  expres¬ 
sion  with  two  parameters,  the  grazing  angle  at  which  shadowing  begins  and  the  maximum 
shadowing  fraction  at  0  degrees.  To  deal  with  the  third  effect  of  RCS  bias,  we  observe  that 
the  monostatic  RCS  of  a  trihedral  falls  off  at  angles  far  from  its  axis,  becoming  nearly  zero  at 
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35  degrees  from  its  axis,  but,  as  angles  approach  45  degrees  (i.e.,  either  near  nadir  or  0-degree 
grazing  views),  the  response  peaks  again,  presumably  due  to  specular  reflection  from  the  edges 
(Sarabandi  and  Chiu,  1996).  We  have  captured  the  gross  trends  of  this  response  in  a  simple 
algebraic  expression. 

We  have  implemented  the  SAR  NIIRS  model  that  incorporates  these  three  effects  and 
applied  them  to  car-like  and  tank-like  targets.  We  found  that  the  results  are  not  sensitive  to 
the  vehicle  dimensions  or  orientation.  It  also  is  evident  that  the  RCS  dependence  overwhelms 
the  shadowing  dependence.  Indeed,  curves  of  NIIRS  versus  grazing  angle  hardly  change  if  we 
turn  off  the  shadowing  effect.  Therefore,  we  will  not  use  it.  The  RCS  and  area  weighting  of 
resolution  on  horizontal  and  vertical  surfaces  lead  to  strong  degradation  in  NIIRS  at  grazing 
angles  of  approximately  8  degrees  and  less.  The  example  above  relates  grazing  angle  to  slant 
range  assuming  a  platform  altitude  of  40,000  ft  and  spotlight  SAR  imagery  at  1  m  resolution. 


Additional  Survivability  Details 

Aural  Detection  Model 

In  order  to  model  and  evaluate  the  aural  detection  properties  of  various  designs,  we  will  utilize 
a  combination  of  two  publicly  available  noise  prediction  packages.  The  first,  the  Aircraft  Noise 
Prediction  Program  (ANOPP),  is  a  set  of  modules  developed  by  the  National  Aeronautics  and 
Space  Administration  for  input  processing,  noise  prediction,  and  output  noise  processing  at  the 
observer  location.  Its  purpose  is  to  predict  the  noise  from  aircraft,  accounting  for  the  effects  of 
the  aircraft  characteristics — engine,  operations,  and  atmospheric  conditions.  The  second  pack¬ 
age  is  called  NOISEMAP  and  consists  of  a  suite  of  programs  that  perform  noise  propagation 
extrapolation  and  noise  contour  calculations.  ANOPP  will  be  used  to  predict  initial  aircraft 
noise  spectra  based  on  the  characteristics  of  each  design.  NOISEMAP  will  then  take  the  pre¬ 
dicted  spectra  and  project  noise  contour  maps  over  a  set  of  standardized  scenarios.  The  results 
of  these  scenarios  will  be  compared  across  aircraft  designs  to  assess  aural  detectability. 

For  unmanned  aircraft  system  aural  detection  assessment,  the  driving  noise  factors  are 
characteristics  of  the  propulsion  systems  rather  than  airframe  noise.  The  models  are  quite 
input  intensive,  making  the  detail  of  the  propulsion  systems  paramount  for  accurate  predic¬ 
tion.  Comprehensive  information  from  engine  manufacturers,  also  known  as  engine  decks, 
as  well  as  detailed  characteristics  of  blade  geometry  and  propeller  design,  will  be  necessary  to 
provide  accurate  inputs  for  reliable  noise  prediction.  The  model  does  not  currently  account  for 
the  effects  of  ducting,  placement  of  engines,  or  airframe  noise.  The  fidelity  of  the  predicted 
estimates  will  increase  with  the  introduction  of  some  of  these  factors  and  more-detailed  input 
data. 

Visual  Detection  Model 

The  visual  detection  model  will  be  used  to  help  determine  the  probability  that  the  RPA  can 
be  detected  by  either  the  unaided  eye  or  simple  optics  (e.g.,  binoculars).  The  model  takes  the 
parametric  description  of  the  airframe  and  creates  a  simple  three-dimensional  model  that  can 
be  projected  on  the  sky  to  give  effective  angular  sizes  and  cross-sections  from  any  distance  or 
viewing  angle.  It  will  then  use  a  set  of  contrast  ratios  to  derive  the  probability  of  visual  detec¬ 
tion  at  any  distance  by  either  the  unaided  eye  or  simple  optics.  The  contrast  ratio  will  capture 
the  color-intensity  difference  between  U.S.  Air  Force  gray  paint  and  the  sky  in  a  variety  of 
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weather  conditions.  At  higher  fidelity,  the  model  will  incorporate  atmospheric  transmission 
information  (provided  by  an  external  program,  such  as  MODTRAN)  and  increased  PDs  for 
moving  objects  in  the  human  visual  held. 

Because  determining  the  intrinsic  contrast  ratio  from  physical  first  principles  is  extremely 
difficult,  the  fidelity  in  the  model  will  be  dictated  by  the  contrast  ratio  calibration.  The  con¬ 
trast  ratio  will  be  calibrated  by  using  data  on  the  maximum  visual  detection  ranges  for  known 
aircraft  that  are  most  similar  to  the  aircraft  being  analyzed. 


Air  Defense  Engagement  Models 

Enhanced  Surface-to-Air  Missile  Simulation 

ESAMS  is  a  one-on-one  missile  engagement-level  model  that  will  be  used  to  determine  suscep¬ 
tibility  to  RF  SAMs.  The  model  includes  aircraft,  missile,  radar,  environment,  and  electronic 
countermeasure  (ECM)  characteristics.  ESAMS  models  the  detection,  tracking,  missile  flyout, 
and  intercept,  resulting  in  a  Pk  against  a  target  aircraft.  The  summary  output  includes  the 
initial  position  from  which  the  missile  is  launched,  the  point  of  closest  approach  between  the 
missile  and  aircraft,  the  miss  distance  (distance  between  the  missile  and  aircraft  when  the  mis¬ 
sile  detonates),  and  a  Pk  value. 

Missile  system  data  are  readily  available  for  most  currently  existing  RF  SAM  systems. 
In  order  to  model  a  SAM  system  not  contained  in  the  standard  ESAMS  input  deck,  specific 
missile  system  characteristics  must  be  provided.  Missile  system  characteristics  include  missile 
fire  control,  aerodynamics,  and  guidance  of  both  ground-  and  missile-seeker  radar.  ECMs  may 
be  also  added.  Though  the  model  may  be  run  in  a  deterministic  mode,  some  effects,  such  as 
multipath,  clutter,  and  noise,  are  modeled  using  ESAMS  in  a  Monte  Carlo  mode.  ESAMS  is 
developed  and  supported  by  Survivability/Vulnerability  Information  Analysis  Center. 

Modeling  System  for  the  Advanced  Investigation  of  Countermeasures 

MOSAIC  simulates  engagements  between  IR-guided  missiles  and  aircraft  with  countermea¬ 
sures.  MOSAIC  simulates  engagements  ranging  from  one-on-one  up  to  few-on-few.  Coun¬ 
termeasures  in  MOSAIC  include  flares  and  IR  jammers,  and  both  preemptive  and  reactive 
countermeasure  strategies  can  be  simulated.  MOSAIC  is  capable  of  simulating  laboratory 
experiments,  field  tests,  and  live-fire  engagements  and  is  part  of  the  Air  Force  Standard  Analy¬ 
sis  Toolkit. 

The  MOSAIC  database  includes  IR  signature  data  for  several  types  of  aircraft,  detailed 
representations  of  numerous  flares  and  jammers,  and  high-fidelity  models  of  several  types  of 
surface-to-air  and  air-to-air  IR  missiles.  MOSAIC  uses  the  MODTRAN  model  to  account  for 
the  effects  of  atmospheric  absorption  and  scattering  of  radiation. 

The  baseline  engagement  configuration  simulates  a  one-on-one  flyout  of  an  IR  missile 
versus  a  countermeasure-equipped  aircraft  flying  a  user-defined  flight  profile.  The  MOSAIC 
flyout  simulation  determines  the  closest  approach  of  the  missile  to  either  the  geometric  cen¬ 
troid  of  the  aircraft  or  its  skin.  The  primary  output  of  MOSAIC  is  the  “hit”  or  “miss”  deter¬ 
mination  for  a  particular  engagement.  It  can  also  generate  time  history  data  pertaining  to  the 
aircraft,  missile  and  missile  seeker,  missile  warning  system,  and  countermeasures.  The  engage¬ 
ment  is  declared  a  “hit”  or  a  “miss”  according  to  a  user-definable  miss  distance  threshold.  Iter¬ 
ating  over  multiple  missile  launch  points  allows  for  the  determination  of  missile  lethal  engage- 
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ment  envelopes  or  “footprints.”  MOSAIC  can  also  simulate  engagements  in  which  multiple 
missiles  of  the  same  type  are  launched  against  an  aircraft  and  in  which  countermeasures  may 
be  carried  on  escort  aircraft. 

Radar-Directed  Gun  Simulation 

RADGUNS  is  used  to  evaluate  the  effectiveness  of  air  defense  artillery  (ADA)  gun  systems 
against  penetrating  aerial  targets.  It  can  also  be  used  to  evaluate  the  effectiveness  of  different 
airborne  target  characteristics  (e.g.,  RCS,  presented  vulnerable  area,  maneuvers,  use  of  ECMs) 
against  a  specific  ADA  system. 

The  output  generated  by  the  program  takes  three  basic  forms — tabular  data  files,  data 
files  for  plotting,  and  graphics  output  files.  An  engagement  simulation  produces  an  event-by¬ 
event  tabular  printout,  listing  the  important  results  of  the  simulation.  These  results  include 
the  time,  target  position,  antenna  boresight  position,  rounds  fired,  single  intercept,  burst,  and 
cumulative  probabilities  of  hit  and  kill,  expected  number  of  hits,  and  some  user-selectable 
simulation  results,  such  as  target  RCS,  range  of  target  at  time  of  intercept,  miss  distance,  and 
tracking  errors. 

The  model  is  high  fidelity,  insofar  as  every  round  fired  is  tracked.  Also,  there  are  high- 
resolution  models  for  ECMs,  tracking,  and  other  engagement  parameters.  Up  to  30  compo¬ 
nents  of  the  aircraft  can  be  represented  using  either  a  six-view  (top,  bottom,  left,  right,  front, 
rear)  or  26-view  presented  vulnerable  area,  with  different  areas  for  four  different  types  of  air¬ 
craft  kills.  RCS  is  defined  in  terms  of  the  roll,  pitch,  and  yaw  of  the  aircraft,  with  different 
tables  for  differing  frequencies  and  antenna  polarizations  of  the  detecting  radar.  Flight  paths 
can  be  linear  with  constant  velocity  or  acceleration  over  a  period  of  time,  nap  of  the  earth, 
sinusoidal  (jinking),  a  CAS  fly-by  (with  a  climb  and  dive  maneuver),  with  turns  or  circular 
paths.  Also,  terrain  can  be  modeled  by  adding  “hills”  that  truncate  parts  of  the  area  from  fire. 
There  are  also  detailed  models  (bring,  detection,  and  tracking)  for  each  of  the  ADA  systems 
that  are  included  with  RADGUNS.  Choices  in  the  weapon  system  conbguration,  target  and 
battleheld  parameters,  and  program  output  are  available.  Examples  are  the  acquisition  radar 
antenna  scan  pattern,  target  track  mode,  target  flight  path,  clutter  and  multipath  parameters, 
and  various  output  data  options. 

Brawler 

Brawler  is  an  air-to-air  combat  model  for  one-on-one  or  few-on-few  airplane  engagements 
that  will  be  used  to  evaluate  RPA  survivability  against  airborne  threats.  Brawler  is  an  event- 
driven,  Monte  Carlo  simulation.  Both  visual  and  BVR  engagements  permit  the  analyst  to 
represent  mission  and  tactical  doctrines,  aggressiveness  of  pilot’s  actions,  perceived  capability 
of  the  enemy,  reaction  time,  and  the  quality  of  the  decisions  made.  Additional  inputs  include 
aircraft,  weapon,  and  sensor  performance  capabilities  for  the  threat  and  Air  Force  systems,  as 
well  as  the  force  sizes,  base  locations,  and  ECM  effects. 

Useful  output  from  Brawler  to  reference  for  SCOPEM  input  is  a  log  of  the  scenario  mod¬ 
eled  that  includes  important  events,  such  as  detections,  weapon  firings,  and  kills.  A  second 
Brawler  output  file  provides  data  for  statistical  calculations  in  the  analysis  of  multiple  runs  and 
would  be  a  source  for  random  draws  from  SCOPEM  given  that  a  similar  scenario  is  explored 
in  the  SCOPEM  mission-level  simulation. 
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